Computer-implemented method, system and computer program (prediction of object deformation using generative adversarial network model)

Abstract

To provide a method, a system and a computer program product for visually generating an object deformation prediction using a generative adversarial network.SOLUTION: A processor may identify, based on an analysis of image data, multiple objects that are in a first stack formation. The processor may determine a load distribution of each object of the multiple objects in relation to a subset of objects of the multiple objects in the first stack formation. The processor may generate, using a Generative Adversarial Network (GAN) algorithm and based on the load distribution for each object, a visualization depicting deformation of each object of the multiple object in relation to the subset of objects of the plurality of objects in the first stack formation. The processor may display the visualization depicting the deformation of each object of the multiple objects to a user.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure generally relates to neural network models, and more specifically to the generation of object deformation prediction using adversarial generative network (GAN) models.

[0002] Deformation refers to a change in the shape, size, or structure of an object due to external forces or conditions. In products such as vegetables, fruits, or packages, various types of deformations can occur, especially when they are stacked in environments such as stores, shopping carts, product shelves, and trucks.

[0003] One common type of deformation is compression deformation. This occurs when products are stacked on top of each other and pressure is applied to the underlying object or objects. In the context of vegetables and fruits, due to the weight of the upper object, the lower object can become flattened, bruised, or even crushed. Similarly, packaged products can undergo compression deformation if they are stacked too high or if heavy objects are placed on top of them. This type of deformation can lead not only to physical damage but also to damage of delicate objects, reducing their quality and shelf life.

[0004] Another type of deformation is bending deformation. This occurs when a product bends or curves due to an uneven distribution of weight or pressure. In the case of objects such as fruits and vegetables, individual bends or curves can occur if they are stacked in a way that does not evenly distribute the load. Additionally, packaged products with weak or inappropriate packaging materials can bend or flex under the weight of the upper object, leading to an unappealing package and potentially damaging the contents. Bending deformation can cause appearance problems, reduce the attractiveness of the product, and further potentially lead to structural weakness.

[0005] In summary, stacking products such as vegetables, fruits, or packages in various environments can lead to different types of deformations. Compressive deformation can occur when an object is crushed by the weight of other objects, while bending deformation involves bending or curving due to uneven pressure distribution. To mitigate these deformations and ensure product quality, appropriate packaging, stacking techniques, and load distribution strategies should be adopted in the processes of storage, transportation, and display.

Summary of the Invention

Problems to be Solved by the Invention

[0006] Deformation-related problems encompass a wide range of harmful consequences, including physical damage, breakage, and compression. When products are stacked without proper consideration of load distribution and structural integrity, they are susceptible to the effects of deformation caused by excessive weight applied from above. This deformation not only impairs the appearance of the product but also poses potential safety risks, especially when handling fragile or perishable objects.

Means for Solving the Problems

[0007] Embodiments of the present disclosure include a method, system, and computer program product for visually generating object deformation prediction using an adversarial generation network. A processor can identify a plurality of objects in a first stack configuration based on analysis of image data. The processor can determine the load distribution of each object of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack configuration. The processor can use an adversarial generation network (GAN) algorithm to generate a visualization depicting the deformation of each object of the subset of the objects among the plurality of objects in the first stack configuration based on the load distribution for each object. The processor can display the visualization depicting the deformation of each object of the plurality of objects to a user.

[0008] The above summary is not intended to describe each of the illustrated embodiments or all implementations of the present disclosure.

Brief Description of the Drawings

[0009] The drawings included in the present disclosure are incorporated into the specification and form a part thereof. They illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. The drawings are merely examples of typical embodiments and do not limit the present disclosure.

[0010]

Figure 1

[0011]

Figure 2A

[0012]

Figure 2B

[0013]

Figure 3

[0014]

Figure 4

[0015]

Figure 5A

[0016]

Figure 5B

[0017]

Figure 5C

[0018]

Figure 6

[0019]

Figure 7

[0020]

Figure 8

[0021] While the embodiments described herein are susceptible to various modifications and alternative forms, details thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the particular embodiments described are not to be construed in a limiting sense. Rather, it is intended to cover all modifications, equivalents, and alternative forms falling within the spirit and scope of the present disclosure. **DETAILED DESCRIPTION**

[0022] Aspects of the present disclosure relate to adversarial generative networks and, more specifically, to the generation of object deformation predictions using a GAN model. While the present disclosure is not necessarily limited to such applications, various aspects of the present disclosure can be understood through discussion of various examples using this context.

[0023] According to an aspect of the present invention, based on the analysis of image data, identifying a plurality of objects in a first stack form; determining the load distribution of each object of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack form; using a Generative Adversarial Network (GAN) algorithm / model, generating a visualization depicting the deformation of each object of the plurality of objects with respect to a subset of the objects in the first stack form based on the load distribution of each object; and displaying to the user a visualization depicting the deformation of each object of the plurality of objects. A computer-implemented method is provided that includes these steps. This is advantageous because the method can be used to generate a visualization of the predicted object deformation that can be used to notify the user that re-stacking and / or repositioning of the objects may be required. Thus, by using object deformation prediction, damage to objects such as food products can possibly be prevented.

[0024] In an embodiment, the method may further include determining a period during which a plurality of objects are in the first stack form; using a GAN algorithm to generate a series of predicted deformation patterns for each object of the plurality of objects over a period of time based on the load distribution of each object; and displaying to the user a series of predicted deformation patterns for each object of the plurality of objects over the period. This is advantageous because the method can display the predicted pattern of object deformation over time, where the user can determine whether repositioning of the objects is necessary.

[0025] In an embodiment, the method may further comprise determining a second stack configuration that minimizes the deformation of each object of a subset of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stack configuration; generating a visualization of the minimized deformation of each object of the plurality of objects in the second stack configuration using a GAN algorithm; and displaying the visualization of the plurality of objects in the second stack configuration to a user. This is advantageous because the method automatically generates a visualization that predicts a new stack configuration that best prevents object deformation.

[0026] In an embodiment, the method may further comprise identifying, by a user, based on analysis of second image data, that a new object has been added to the plurality of objects configured in the second stack configuration; evaluating the impact of the new object on the load distribution of each object of the plurality of objects in the second stack configuration; determining a location for placing the new object in the second stack configuration, where the location for placing the new object minimizes the deformation of each object with respect to a subset of the objects in the second stack configuration; generating a visualization of the location for placing the new object within the second stack configuration using a GAN algorithm; and displaying the visualization of the location for placing the new object within the second stack configuration. This is advantageous because the method automatically determines where to position a new object added to the stack configuration based on the predicted load distribution and / or deformation.

[0027] In an embodiment, the step of generating a visualization depicting the deformation of each object of a plurality of objects includes the step of analyzing the packaging specifications of one or more of the plurality of objects. This is advantageous because, by this method, based on known attributes associated with the type of packaging (e.g., the type of packaging material, shape, etc.), load distribution and / or object deformation prediction can be performed based on known attributes associated with the type of packaging (e.g., the tensile strength of the packaging).

[0028] In an embodiment, determining the load distribution of each object of a plurality of objects is based on a set of object attributes associated with a product identification code identified through analysis of image data. This is advantageous because, by this method, based on known attributes associated with the type of object (e.g., the type of product / object, susceptibility to damage, etc.) using a product identification code (e.g., a barcode), an accurate load distribution and / or object deformation prediction can be performed.

[0029] In an embodiment, the step of determining the load distribution of each object of a plurality of objects for a subset of the objects in a first stack configuration includes the step of determining a dynamic load distribution based on the movement applied to the first stack configuration. This is advantageous because, in this method, the load distribution prediction can be based on various movements that can occur to the objects due to user interaction (e.g., the objects can move when pushing a shopping cart filled with food products).

[0030] In an embodiment, the image data is generated by an Internet of Things (IoT) camera. In this method, one or more IoT cameras that can be used to generate an image of a given object are utilized.

[0031] In an embodiment, the visualization depicting the deformation of each of a plurality of objects is displayed to a user via augmented reality glasses. This is advantageous because the object deformation prediction and / or the generated stack form can be visually displayed to the user via the AR glasses. Thus, the user can be immediately informed of any potential object deformation while continuing to shop.

[0032] According to an aspect of the present invention, there is provided a system comprising a processor and a computer-readable storage medium communicatively coupled to the processor and storing program instructions that cause the processor to execute a method when executed by the processor. The method executed by the processor includes: identifying a plurality of objects in a first stack form based on analysis of image data; determining a load distribution of each of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack form; using an adversarial generation network (GAN) algorithm to generate a visualization depicting the deformation of each of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack form based on the load distribution of each object; and displaying the visualization depicting the deformation of each of the plurality of objects to the user. This is advantageous because the system can be used to generate a visualization of the predicted object deformation that can be used to notify the user that object restacking and / or rearrangement may be required. Thus, by using object deformation prediction, damage to objects such as food items can likely be prevented.

[0033] In an embodiment, the method executed by the system's processor may further comprise determining a period during which a plurality of objects are in a first stack form; using a GAN algorithm to generate a series of predicted deformation patterns for each of the plurality of objects over a period of time based on the load distribution of each object; and displaying to the user the series of predicted deformation patterns for each of the plurality of objects over the period. This is advantageous because the system can display the predicted pattern of object deformation over time, where the user can determine whether object rearrangement is necessary.

[0034] In an embodiment, the method executed by the system's processor may further comprise determining a second stack form that minimizes the deformation of each of the plurality of objects with respect to a subset of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stack form; using a GAN algorithm to generate a visualization of the minimized deformation of each of the plurality of objects in the second stack form; and displaying to the user the visualization of the plurality of objects in the second stack form. This is advantageous because the system automatically generates a visualization that predicts a new stack form that best prevents object deformation.

[0035] In an embodiment, the method executed by the system's processor includes: identifying, based on the analysis of the second image data, that a new object has been added by the user to a plurality of objects configured in a second stack form; evaluating the impact of the new object on the load distribution of each object among the plurality of objects in the second stack form; determining a position for placing the new object in the second stack form, where the position for placing the new object minimizes the deformation of each object with respect to a subset of the objects in the second stack form; generating a visualization of the position for placing the new object within the second stack form using a GAN algorithm; and further comprising displaying the visualization of the position for placing the new object within the second stack form. This is advantageous because the system automatically determines where to position the new object added to the stack form based on the predicted load distribution and / or deformation.

[0036] In an embodiment, the step of generating a visualization depicting the deformation of each object among the plurality of objects includes analyzing the packaging specifications of one or more of the plurality of objects. This is advantageous because the system can use the packaging specifications (e.g., type and shape of the packaging material) to predict the load distribution and / or object deformation based on known attributes associated with the type of packaging (e.g., tensile strength of the packaging).

[0037] In an embodiment, determining the load distribution of each object among the plurality of objects is based on a set of object attributes associated with a product identification code identified through the analysis of the image data. This is advantageous because the system can use the product identification code (e.g., barcode) to accurately predict the load distribution and / or object deformation based on known attributes associated with the type of object (e.g., type of product / object, susceptibility to damage, etc.).

[0038] In an embodiment, the step of determining the load distribution of each object of a plurality of objects with respect to a subset of objects of the plurality of objects in the first stack form includes the step of determining a dynamic load distribution based on the movement applied to the first stack form. This is advantageous because in the system, the load distribution prediction can be based on various movements that can occur to the objects due to user interactions (e.g., an object can move when pushing a shopping cart filled with groceries).

[0039] According to an aspect of the present invention, there is provided a computer program product including a computer-readable storage medium in which program instructions are embodied, the program instructions being executable by a processor to cause the processor to execute a method. The program instructions executable by the processor include: identifying a plurality of objects in a first stack form based on an analysis of image data; determining the load distribution of each object of a plurality of objects with respect to a subset of objects of the plurality of objects in the first stack form; generating a visualization depicting the deformation of each object of a plurality of objects with respect to a subset of objects of the plurality of objects in the first stack form based on the load distribution of each object using an adversarial generation network (GAN) algorithm; and displaying the visualization depicting the deformation of each object of the plurality of objects to a user. This is advantageous because the computer program product can be used to generate a visualization of the predicted object deformation that can be used to notify the user that object restacking and / or rearrangement may be required. Thus, by using object deformation prediction, damage to objects such as groceries can possibly be prevented.

[0040] In an embodiment, the program instructions executable by a processor may further include: determining a period during which a plurality of objects are in a first stack form; using a GAN algorithm to generate a series of predicted deformation patterns for each of the plurality of objects over a period of time based on the load distribution of each object; and displaying the series of predicted deformation patterns for each of the plurality of objects over the period to a user. This is advantageous because the computer program product can display the predicted pattern of object deformation over time, where the user can determine whether object rearrangement is necessary.

[0041] In an embodiment, the program instructions executable by a processor may further include: determining a second stack form that minimizes the deformation of each of the plurality of objects with respect to a subset of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stack form; using a GAN algorithm to generate a visualization of the minimized deformation of each of the plurality of objects in the second stack form; and displaying the visualization of the plurality of objects in the second stack form to the user. This is advantageous because the computer program product automatically generates a visualization that predicts a new stack form that best prevents object deformation.

[0042] In an embodiment, the program instructions executable by the processor include: identifying, based on the analysis of the second image data, that a new object has been added by the user to a plurality of objects configured in a second stack form; evaluating the impact of the new object on the load distribution of each object among the plurality of objects in the second stack form; determining a position for placing the new object in the second stack form, where the position for placing the new object minimizes the deformation of each object with respect to a subset of the objects in the second stack form; generating a visualization of the position for placing the new object within the second stack form using a GAN algorithm; and further comprising displaying the visualization of the position for placing the new object within the second stack form. This is advantageous because the computer program product automatically determines where to position the new object added to the stack form based on the predicted load distribution and / or deformation.

[0043] In an embodiment, the step of generating a visualization depicting the deformation of each object among the plurality of objects includes analyzing the packaging specifications of one or more of the plurality of objects. This is advantageous because the computer program product can use the packaging specifications (e.g., type of packaging material, shape, etc.) to predict the load distribution and / or object deformation based on known attributes associated with the type of packaging (e.g., tensile strength of the packaging).

[0044] In an embodiment, determining the load distribution of each object among the plurality of objects is based on a set of object attributes associated with a product identification code identified through the analysis of the image data. This is advantageous because the computer program product can use the product identification code (e.g., barcode) to accurately predict the load distribution and / or object deformation based on known attributes associated with the type of object (e.g., type of product / object, susceptibility to damage, etc.).

[0045] In an embodiment, the step of determining the load distribution of each object of a plurality of objects with respect to a subset of objects of the plurality of objects in the first stack form includes the step of determining a dynamic load distribution based on the movement applied to the first stack form. This is advantageous because in this computer program product, the load distribution prediction can be based on various movements that can occur to an object due to user interaction (for example, an object can move when pushing a shopping cart filled with groceries).

[0046] In many cases, the load-bearing properties of products, etc.; stacking objects has led to significant problems of deformation, including damage, breakage, and compression. This deformation phenomenon occurs when a product is subjected to external pressure from the load of the stack. This problem is widely seen in scenarios where stacking is a common practice, especially during storage, transportation, and display. The problem of deformation not only results in economic losses for manufacturers, distributors, and retailers due to product damage and a decrease in market value, but also affects consumer satisfaction.

[0047] Embodiments of the present disclosure include a method, system, and computer program product for visually generating object deformation prediction using a GAN model / algorithm. The system recognizes object / product details and the stacking order within an environment (e.g., shopping cart, container, basket, shelf unit, etc.). By estimating the stack load distribution and using a GAN model, the system generates a visualization showing potential deformations to different products on the stack due to the stack, enabling the user to pre-relocate the objects. It adopts a GAN model to predict the deformation pattern considering the load and time duration, and creates animated visualizations of various products over time. The system analyzes the product list to determine the optimal stack that minimizes deformation and uses the GAN model to show potential deformations. When the user stacks an object, the system evaluates the impact of the new object on the load distribution and potential deformations. The system also considers packaging and uses the GAN model to determine whether the packaging can deform and affect the enclosed products. This comprehensive approach improves the stack and prevents damage.

[0048] In an embodiment, when products are stacked in a shopping cart or on a shelf, a camera integrated into the user's AR glasses or an IoT system recognizes these products along with their specifications and stacking order. Subsequently, the proposed system estimates the distribution of loads on various products in the stack. Utilizing a GAN model, the system pre-generates a visual representation showing potential deformations caused by the load distribution. This enables the user to re-locate the products before completing the stacking process and prevents any potential product deformations.

[0049] In an embodiment, the system takes into account the estimated load distributed among various products in the stack and the duration of the applied load. Using a GAN model, the system generates a series of predicted deformation patterns for different products over a specified time frame. These patterns are compiled into an animated visualization, showing the progressive deformation of various products affected by the stack load over a given duration.

[0050] In an embodiment, the proposed system analyzes a consolidated list of products intended for the stack, such as a shopping list or the requirements of a warehouse stack. This analysis determines the optimal stack arrangement that minimizes product deformation, considering the specifications of the objects and other relevant factors. The system visually depicts potential product deformations by employing a GAN model using the recommended stack configuration.

[0051] In an embodiment, when the user starts stacking products, the system continuously evaluates the impact of each newly added object on the load distribution of the stack. It assesses how the addition of each object can affect the deformation in other products within the stack. The GAN model is utilized to show the progressive deformation of products resulting from the addition of new objects to the trolley or shelf.

[0052] In an embodiment, the system considers the packaging specifications for the objects / products enclosed within the package. The system determines whether the package may be susceptible to the effects of deformation due to the stack load across various products with packaging. It also evaluates whether a deformed package can affect the integrity of the contained products.

[0053] In an embodiment, the system identifies the direction of movement of a shopping cart and predicts potential dynamic loads that can affect various products in the stack. The impact of dynamic loads due to movement on inclined or uneven surfaces on product deformation is analyzed. When generating GAN-based predictions, the system takes into account the types and durations of these dynamic loads for different products.

[0054] In an embodiment, communication between shopping carts / trolleys on the floor enables the sharing of movement patterns, dynamic load information, and relative positions. Based on these inputs, the system predicts the movement path of the trolley and pre-indicates potential deformations that may occur due to the expected dynamic load path.

[0055] In the physical shopping area where customers pass through aisles to fill their carts, a significant challenge is to ensure the safety and condition of the stacked products. The aim of this innovative idea is to transform the shopping experience by providing predictive visualization of potential product deformation when stacking objects within a physical cart. By leveraging predictive technologies, shoppers can gain insights, strategically organize their purchases, and visualize the potential outcomes of their stack selections. This enables shoppers to optimize the composition of their carts and protect the quality and integrity of the objects they have selected throughout the shopping journey.

[0056] The above advantages are examples of advantages, and not all advantages have been described. Furthermore, embodiments of the present disclosure can include all, some, or none of the above advantages while remaining within the spirit and scope of the present disclosure.

[0057] Referring now to FIG. 1, a block diagram of an exemplary object deformation analysis system 100 according to an embodiment of the present disclosure is shown. In the illustrated embodiment, the object deformation analysis system 100 includes an object deformation analysis device 102 communicatively coupled to an augmented reality (AR) device 120 and an Internet of Things (IoT) device 130 via a network 150. The object deformation analysis device 102, the AR device 120, and the IoT device 130 may be configured as any type of computer system and may be substantially similar to the computer system 601 described in detail in FIG. 6. In an embodiment, the AR device 120 may include a camera 122 that generates image data 126 of an object to be analyzed for deformation and / or load distribution within a given environment. The AR device 120 may include a display 124 on which various image data (e.g., visualization of an object, a new stack configuration, and / or an object deformation simulation) may be presented to a user.

[0058] The network 150 may be any type of communication network, such as a wireless network or a cloud computing network. The network 150 may be substantially similar to or identical to the computing environment 700 described in FIG. 7. In some embodiments, the network 150 may be implemented within a cloud computing environment or using one or more cloud computing services. Consistent with various embodiments, a cloud computing environment may include a network-based distributed data processing system that provides one or more cloud computing services. Further, a cloud computing environment may include many computers (e.g., hundreds or thousands of computers or more) that are located within one or more data centers and are configured to share resources via the network 150.

[0059] In the illustrated embodiment, the object deformation analysis device 102 includes a network interface (I / F) 104, a processor 106, a memory 108, an object detection component 110, a deep neural network model (DNN) 112, an adversarial generation network (cGAN) model 114, a visual component 116, and an object corpus 118.

[0060] In an embodiment, the object detection component 110 is designed to distinguish and classify a plurality of objects within the image data 126 received from the AR device 120. The image data 126 may be derived from an IoT device such as an IoT device 130 that functions as an additional camera in the environment to capture images of various objects. This component utilizes the You Only Look Once (YOLO) image detection and recognition algorithm to identify various objects within the image data 126. The YOLO methodology involves dividing the image into a grid to enable prediction of bounding boxes and object classes in a single pass, ensuring fast and efficient detection. The adoption of anchor boxes to predict object sizes and the connection of YOLO to a convolutional neural network that processes image features employ non-maximum suppression to refine the predictions and make them suitable for real-time applications in image and video analysis. Its unique approach significantly develops the object detection ability.

[0061] In an embodiment, the object detection component 110 scans the contents of an environment or a container (e.g., a shopping cart) and differentiates and classifies the objects stacked therein. This component utilizes advanced image analysis techniques such as YOLO to accurately identify each object and determine their spatial relationships within the environment. Once an object is identified, the component interacts with an object corpus 130 (e.g., a store database) to further classify the object based on known attributes retrieved from the object corpus 118. These attributes encompass various details including product type, unique packaging, weight, and dimensions.

[0062] In an embodiment, the DNN model 112 is adjusted to calculate the center of gravity position for the identified stacked objects (e.g., products within a shopping cart). It generates a numerical representation that identifies the position along the length of the cart where the combined weight of the products achieves a perfect balance. This load distribution metric indicates the position of the center of gravity of the products in the cart and provides insights regarding potential instability. By having specific load distribution metrics such as the center of gravity position, the DNN model 112 quantitatively shows how the weight of each object is distributed within the environment (e.g., a shopping cart) and may employ a convolutional neural network architecture.

[0063] In an embodiment, the GAN model 114 utilizes the predicted center of gravity values to represent cart stability, along with the real-time image data 126 of the analyzed stack of objects. These combined inputs function as conditions for the GAN model 114. It uses these conditions to generate a visual representation showing the distribution of loads within the environment (e.g., a shopping cart). The output of the GAN model provides a clear depiction of the spatial arrangement of the objects and enhances the shopping experience by providing useful insights into the load distribution. In an embodiment, the visual component 116 can output and generate one or more visualizations that can be presented to the user via the AR device 120 using the GAN model 114. The visualizations can incorporate any potential product deformations into the generated visuals and provide an inclusive perspective. Thus, by fusing refined conditions and complex neural processing, the shopping experience is enriched with detailed insights into both load distribution and potential anomalies.

[0064] In some embodiments, the object deformation analysis device 102 may continuously execute iterations of an experiment using machine learning to generate additional useful training data. For example, when a new set of inputs (such as new data inputs collected / received after implementing an optimized training solution for the current state of the system 100) is presented to the machine learning model, it may determine the type of training based on past actions for similar inputs. As the training data increases, the machine learning model is periodically retrained and / or refactored, and as a result, the accuracy of predicting valid configuration parameter values that may affect the performance metrics of the GAN model based on the predicted changes in the object deformation pattern increases. The results of previous experiments are used to determine configuration and / or workload attribute variations and / or the type of training for collecting data for future experiments. For example, through the use of machine learning, one or more experimental values for one or more configuration parameters can be identified based on the determination that past changes to the deformation pattern of one or more objects have affected one or more performance metrics by more than a threshold amount of change. For example, the machine learning model can identify past changes to object deformation patterns / parameters based on a given training selection and optimize such parameters over time.

[0065] The machine learning model is trained using a specific machine learning algorithm. Once trained, inputs are applied to the machine learning model to make predictions, which may also be referred to herein as predicted outputs or outputs. The machine learning model includes a model data representation or model artifact. The model artifact includes parameter values that may be referred to herein as theta values, which are applied to the inputs by the machine learning algorithm to generate the predicted output. Training the machine learning model involves determining the theta values of the model artifact. The structure and composition of the theta values depend on the machine learning algorithm.

[0066] In supervised training, the training data is used by a supervised learning algorithm to train a machine learning model. The training data includes inputs and “known” outputs. In an embodiment, the supervised learning algorithm is an iterative procedure. In each iteration, the machine learning algorithm applies the model artifact and the inputs to generate a predicted output. An error or distribution between the predicted output and the known output is calculated using an objective function. In effect, the output of the objective function indicates the accuracy of the machine learning model based on a particular state of the model artifact in the iteration. By applying an optimization algorithm based on the objective function, the theta value of the model artifact is adjusted. An example of an optimization algorithm is gradient descent. The iteration may be repeated until a desired accuracy is achieved or until some other criteria are met.

[0067] In a software implementation, when a machine learning model is said to receive an input, execute, and / or generate an output or prediction, a computer system process, such as the object deformation analysis device 102 that executes the machine learning algorithm, applies the model artifact to the input to generate a predicted output. The computer system process executes the machine learning algorithm by executing software configured to cause the algorithm to execute.

[0068] In some embodiments, feature synthesis may be performed. Feature synthesis is a process that converts raw input into features that can be used as input to a machine learning model. Feature synthesis may also convert other features into input features. Feature engineering refers to the process of identifying features. The goal of feature engineering is to identify a set of features with higher feature prediction quality for a machine learning algorithm or model. With features having higher prediction quality, machine learning algorithms and models produce more accurate predictions. In addition, a set of features with high prediction quality tends to be smaller and requires less memory and storage for storage. A set of features with higher prediction quality also enables the generation of machine learning models with lower complexity and smaller artifacts, thereby reducing the training time and execution time when executing the machine learning model. Smaller artifacts also require less memory and / or storage for storage.

[0069] In some embodiments, the object deformation analysis device 102 may utilize machine learning and / or deep learning, where the algorithm or model may be generated by performing supervised, unsupervised, or semi-supervised training on past data inputs and / or past features. Machine learning algorithms may include, but are not limited to, decision tree learning, correlation rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, similarity / metric learning, sparse dictionary learning, genetic algorithms, rule-based learning, and / or other machine learning techniques.

[0070] For example, machine learning algorithms may utilize one or more of the following exemplary techniques, namely, K-Nearest Neighbor (KNN), Learning Vector Quantization (LVQ), Self-Organizing Map (SOM), Logistic Regression, Ordinary Least Squares Regression (OLSR), Linear Regression, Stepwise Regression, Multivariate Adaptive Regression Splines (MARS), Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO), Elastic Net, Least Angle Regression (LARS), Probabilistic Classifier, Naive Bayes Classifier, Binary Classifier, Linear Classifier, Hierarchical Classifier, Canonical Correlation Analysis (CCA), Factor Analysis, Independent Component Analysis (ICA), Linear Discriminant Analysis (LDA), Multidimensional Scaling (MDS), Non-Negative Matrix Factorization (NMF), Partial Least Squares Regression (PLSR), Principal Component Analysis (PCA), Principal Component Regression (PCR), Sammon Mapping, t-Distributed Stochastic Neighbor Embedding (t-SNE), Bootstrap Aggregating, Harmonic Mean, Gradient Boosting Decision Tree (GBDT), Gradient Boosting Machine (GBM), Inductive Bias Algorithm, Q-Learning, State-Action-Reward-State-Action (SARSA), Temporal Difference (TD) Learning, Apriori Algorithm, Equivalence Class Transformation (ECLAT) Algorithm, Gaussian Process Regression, Gene Expression Programming, Group Method of Data Handling (GMDH), Inductive Logic Programming, Example-Based Learning, Logical Model Tree, Information Fuzzy Network (IFN), Hidden Markov Model, Gaussian Naive Bayes, Multinomial Naive Bayes, Averaged One-Dependence Estimator (AODE), Bayesian Network (BN), Classification and Regression Tree (CART), Chi-Square Automatic Interaction Detection (CHAID), Expectation Maximization Algorithm, Feed-Forward Neural Network, Logic Learning Machine, Self-Organizing Map, Single Linkage Clustering, Fuzzy Clustering, Hierarchical Clustering, Boltzmann Machine, Convolutional Neural Network, Recurrent Neural Network, Hierarchical Temporal Memory (HTM), and / or one or more of other machine learning techniques.

[0071] FIG. 1 is intended to depict representative major components of an object deformation analysis system 100. However, in some embodiments, individual components may have a higher or lower complexity than those represented in FIG. 1, components other than those shown in FIG. 1, or components in addition thereto may exist, and the number, type, and configuration of such components may vary. Similarly, one or more components shown with the object deformation analysis system 100 may not exist, and the arrangement of the components may vary. For example, FIG. 1 shows an exemplary object deformation analysis system 100 having a single object deformation analysis device 102, a single AR device 120, and a single IoT device 130 communicatively coupled via a single network 150, but a suitable network architecture for implementing embodiments of the present disclosure may include any number of object deformation analysis devices, AR devices, IoT devices, and networks. The various models, modules, systems, and components shown in FIG. 1, if present, may exist across multiple object deformation analysis devices, AR devices, IoT devices, and networks.

[0072] Referring now to FIG. 2A, there is shown 202 an exemplary view of a plurality of objects 204 within a shopping cart 200, according to an embodiment of the present disclosure. It should be noted that this represents only one example of the analysis and prediction of object deformation in the context of grocery shopping and should not be considered limiting. When a user begins their shopping journey, it typically starts with selecting a plurality of objects 204 from a store's offerings. In the depicted scenario, a system such as the object deformation analysis system 100 employs an automatic product detection and location recognition algorithm to identify the plurality of objects 204 within the shopping cart 200 as shown in the shopping cart diagram 202.

[0073] In certain implementations, this system may utilize the YOLO (You Only Look Once) object detection algorithm, an innovative architecture in computer vision. YOLO functions by dividing an image into a grid and simultaneously predicting bounding boxes and object classes, ensuring speed and efficiency. The figure shows bounding boxes around objects 204A, 204B, and 204C. Prediction of object size is assisted by anchor boxes, while the convolutional neural network processes image features. YOLO also uses non-maximum suppression to refine predictions, making it suitable for real-time applications such as image and video analysis. The unique approach of its single-pass and grid-based methodology significantly improves object detection capabilities.

[0074] In these embodiments, the system examines the contents of shopping cart 200 and identifies and verifies pre-stacked objects 204. Through sophisticated image analysis utilizing YOLO, each object is accurately recognized and its relative position within the cart is determined. This automated process not only increases efficiency but also sets the foundation for optimizing the placement of objects based on the existing configuration, ensuring a harmonious and well-organized layout. As a result of YOLO's computational power, a comprehensive and understandable depiction of the cart's contents is provided, with bounding boxes visually presented for each detected object. These boxes, supplemented by labels, clearly identify the recognized objects or products, providing the user with a clear overview of their current shopping cart configuration.

[0075] Referring now to FIG. 2B, an exemplary attribute table 206 associated with the identified object according to an embodiment of the present disclosure is shown. In an embodiment, when an object is identified using an automatic product detection and location recognition algorithm (e.g., YOLO), the system can interact with a database to classify the identified object based on known attributes. For example, the system can access a grocery store database that includes details for each of the identified objects or products, including their unique packaging, weight, and dimensions. The system can use these attributes to generate an attribute table 206 that classifies the identified objects. For example, the product type, weight, type, packaging type, and strength attributes of the object can be listed in the table. The system can analyze these attributes when generating an object deformation prediction. For example, an object such as bread, which has no packaging material and has a strength type of "easily damaged," is likely to deform. On the other hand, an object such as a water bottle, which is packaged and not easily damaged, is resistant to deformation. The system can use these attributes to accurately determine and generate various visualizations related to the object deformation prediction.

[0076] Referring now to FIG. 3, there is shown an exemplary diagram for determining load distribution using a deep neural network (DNN) model 300 according to an embodiment of the present disclosure. In an embodiment, the deep neural network 300 calculates the centroid position for a plurality of objects in the shopping cart 302. In an embodiment, the DNN 300 can account for various angles of the shopping cart 302 (e.g., ground slope, cart tilt, etc.) and / or the position of the objects (based on various views of the objects) when determining the load distribution. The DNN 300 outputs a numerical value representing the position along the length of the shopping cart 302, where the combined weights of the objects are perfectly balanced. This load distribution metric indicates that the centroid of the objects / products in the shopping cart is close to the rear of the cart, thereby potentially destabilizing the cart slightly. By providing a specific load distribution metric such as the centroid position, the DNN 300 provides quantitative insights into how the weights are distributed in the shopping cart 302. This information helps the user make an informed decision about how to arrange the products for better stability and balance during transportation. It functions as a guideline for optimizing the arrangement of the objects and ensures a safer and more stable shopping cart 302 configuration.

[0077] In some embodiments, the DNN 300 can be a convolutional neural network (CNN) consisting of multiple layers that typically cooperate to process and analyze visual data. The first layer, known as the input layer, receives raw image data. The convolutional layers perform feature extraction by convolving learned filters over the input to capture different patterns. The activation function introduces non-linearity and improves the network's ability to model complex relationships. The pooling layer downsamples the extracted features to reduce computational complexity and improve robustness. The fully connected layer at the end of the network integrates the extracted features for the final prediction, and the CNN becomes a powerful tool for image recognition and analysis. In this scenario, the load distribution metric: the center of gravity value is predicted.

[0078] Referring now to FIG. 4, an exemplary diagram for predicting object deformation using an adversarial generative network (GAN) model 400 according to an embodiment of the present disclosure is shown. In some embodiments, the GAN model 400 can be a conditional GAN model. In an embodiment, the GAN model 400 is applied to the output of FIG. 3 to generate a load distribution visualization. The predicted center of gravity value, which indicates the balance of the shopping cart 402, is concatenated with real-time image data of the stack / configuration / arrangement of the objects within the cart. These combined inputs function as conditions for the GAN model 400. The network then utilizes these conditions to generate a visual representation of how the load is distributed within the shopping cart 402. The output of the GAN model provides a clear illustration of the spatial arrangement of the objects, enhancing the shopping experience with valuable insights into the load distribution. It incorporates any potential product deformations into the generated visuals, providing a comprehensive perspective that encompasses everything. By fusing such refined conditions and complex neural processing, a comprehensive view is provided, enriching the shopping experience with detailed insights into both load distribution and potential anomalies.

[0079] In the illustrated embodiment, the GAN model 400 is a sophisticated machine learning architecture designed to generate data based on specific conditions. The GAN model 400 comprises two main components, a generator and a discriminator, and the GAN performs game-like dynamics. The generator takes in random noise and conditions (such as the predicted load distribution) and creates data that mimics the desired output (such as a visual representation of the stack load distribution). On the other hand, the discriminator evaluates the generated data against real-time data, promoting a competitive learning process. As a result of this adversarial interaction, the generator improves at generating highly realistic outputs, and the GAN model 400 becomes a powerful tool for data generation tasks under specified conditions. Thus, the GAN model is configured to generate a shopping cart visualization that can be displayed to a user via one or more display devices (e.g., AR glasses, computer screen, etc.).

[0080] In some embodiments, the GAN model 400 may include GAN-Enhanced Cart Regeneration with Visual and Heat Map Insights. This versatile system plays a dual role in cart regeneration. In an embodiment, the GAN model 400 employs its capabilities to generate a visually enhanced shopping cart. In an embodiment, the GAN model 400 generates a graphical overlay that intricately displays the visualization of the stack load distribution of objects, depicting a clear picture of how various objects are arranged. This visual enhancement provides the user with a clear understanding of the load distribution within the cart.

[0081] In some embodiments, the GAN model 400 may generate a heatmap representation of the stack load distribution. The dynamic heatmap may display to the user a visualization of the intensity of the load across different regions, providing comprehensive and intuitive insights into potential pressure points and load concentrations. Thus, the unique interaction between the visual and heatmap of the GAN model improves cart regeneration and enables the user to gain a multifaceted understanding of the load distribution. This symbiotic relationship between the creative generation of the GAN model and the reconstruction of the cart adds a layer of depth of information to the shopping experience.

[0082] Thus, the user can make decisions about the stacking and / or placement of objects with knowledge of the load distribution. After viewing both the visual and heatmap representations, the user can understand the dynamics of the load in the shopping cart. With this understanding, the user can pre-select any product for potential rearrangement. This selection enables the user to fine-tune the load distribution, optimize space utilization, and create a personalized and well-organized cart arrangement. This user-driven interaction, influenced by data-driven insights, harmonizes technology and user agency and improves the overall shopping experience.

[0083] Referring now to FIG. 5A, an exemplary first stack configuration 500A according to an embodiment of the present disclosure is shown. In the illustrated embodiment, the image data representing the first stack configuration 500A includes a pack of water bottles placed on top of a packed apple pie. The system may use the attribute table 206 from FIG. 2B to analyze the image data and identify that the packed apple pie is packed in a thin box that is easily damaged. Further, based on the weight of the water bottles, the system may determine that if this first stack configuration remains as is, the packed apple pie will be deformed (shown in FIG. 5B) by the water bottles.

[0084] Referring now to FIG. 5B, an example of an object deformation 500B according to an embodiment of the present disclosure is shown. In an embodiment, the system may generate a visualization of an object based on a predicted deformation. As shown, the system generates a visualization depicting the deformation of the apple pie packaging due to the weight of the water bottle. This visualization may be presented to the user via AR glasses or some other display type (e.g., a computer screen).

[0085] Referring now to FIG. 5C, an example of a predicted second stack configuration 500C according to an embodiment of the present disclosure is shown. In an embodiment, the system may generate a visualization of a second stack configuration 500C that minimizes the deformation of a given object. As shown, the system relocates the objects by placing the easily damaged apple pie on top of the heavier water bottle, as shown in the second stack configuration 500C. The second stack configuration may be shown in the visualization presented to the user. Thus, the system may predict a better stack configuration that can be used or implemented by the user to reduce any deformation of a given object.

[0086] Referring now to FIG. 6, an exemplary process 600 for generating object deformation predictions using a GAN model according to some embodiments of the present disclosure is shown. Process 600 may be performed by processing logic comprising hardware (e.g., circuits, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processor), firmware, or any combination thereof. In some embodiments, process 600 is a computer-implemented process. In an embodiment, process 600 may be performed by the processor 106 of the object analysis device 102 illustrated in FIG. 1.

[0087] In an embodiment, the proposed system estimates the stack load distribution on different objects or products based on their specifications and subsequent objects placed on the stack. The system utilizes an AR glass or a display system to show how various objects / products can deform due to the applied stack load, and the present invention uses a GAN to create a visualization of how the product deforms due to the stack load distribution, so that as a result, the user can take actions in advance to prevent the deformation of the product.

[0088] In an embodiment, process 600 starts by identifying a plurality of objects in a first stack configuration based on the analysis of image data. This is shown in step 505. In an embodiment, the image data is generated by an Internet of Things (IoT) camera.

[0089] Process 600 continues by determining the load distribution of each object of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack configuration. This is shown in step 610.

[0090] In some embodiments, determining the load distribution of each object of the plurality of objects is based on a set of object attributes associated with a product identification code identified through the analysis of image data.

[0091] In some embodiments, the step of determining the load distribution of each object of the plurality of objects with respect to a subset of the objects among the plurality of objects in the first stack configuration includes the step of determining a dynamic load distribution based on the movement applied to the first stack configuration.

[0092] Process 600 continues to use a Generative Adversarial Network (GAN) algorithm to generate a visualization depicting the deformation of each object of a subset of objects among a plurality of objects in a first stack configuration based on the load distribution for each object. This is shown at step 615. In some embodiments, the step of generating a visualization depicting the deformation of each object of a plurality of objects includes the step of analyzing the packing specifications of one or more of the plurality of objects.

[0093] Process 600 continues to display to the user a visualization depicting the deformation of each object of a plurality of objects. This is shown at step 620. In some embodiments, the visualization depicting the deformation of each object of a plurality of objects is displayed to the user via augmented reality glasses.

[0094] In some embodiments, process 600 may comprise the steps of determining a period during which a plurality of objects are in a first stack configuration; using a GAN algorithm to generate a series of predicted deformation patterns for each object of a plurality of objects over a period of time based on the load distribution for each object; and displaying to the user the series of predicted deformation patterns for each object of a plurality of objects over the period.

[0095] In some embodiments, process 600 may comprise the steps of determining a second stack configuration that minimizes the deformation of each object of a plurality of objects with respect to a subset of objects of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stack configuration; using a GAN algorithm to generate a visualization of the minimized deformation of each object of a plurality of objects in the second stack configuration; and displaying to the user the visualization of the plurality of objects in the second stack configuration.

[0096] When object deformation prediction is performed, the user can take actions based on the provided suggestions. With precise guidance from the neural network, the user has the autonomy to rearrange the products in the cart. Through this interactive process, the user can apply the suggested adjustments, optimize the load distribution, and maximize the space efficiency. By seamlessly converting the findings of the prediction into actionable changes, the user actively participates in the fine-tuning of the arrangement of their cart. As a result of this dynamic collaboration between technology-driven suggestions and user-driven adjustments, the shopping experience is improved, reflecting a harmonious integration of intelligent guidance and individual choices.

[0097] In some embodiments, process 600 includes identifying, by the user, based on the analysis of the second image data, that a new object has been added to a plurality of objects configured in a second stack configuration; evaluating the impact of the new object on the load distribution of each object of the plurality of objects in the second stack configuration; determining a position for placing the new object in the second stack configuration, where the position for placing the new object minimizes the deformation of each object with respect to a subset of the objects in the second stack configuration; generating a visualization of the position for placing the new object within the second stack configuration using a GAN algorithm; and displaying the visualization of the position for placing the new object within the second stack configuration.

[0098] In this way, the sophisticated deep neural network serves as a prediction guide, providing precise guidance for repositioning the selected products within the cart. Utilizing its advanced pattern recognition capabilities, this neural network accurately predicts the bounding box coordinates of the ideal positions within the cart for the repositioning of the selected products. The insights from this prediction help the user make an informed decision and ensure fine-tuning to optimize load distribution and space utilization. By seamlessly merging advanced technology with user interaction, this neural network enhances the shopping experience through adjusted and efficient product placement.

[0099] The system utilizes smart technology involving user participation to create a better shopping cart experience. By using advanced systems such as convolutional neural networks and adversarial generative networks, it shows the user how products are stacked and suggests changes for a better balance. The user can refer to the load details to adjust the product positions and make the cart more efficient. This collaboration between technology and user input aims to improve cart stability, protect products, and enable the user to shop more wisely.

[0100] Referring now to FIG. 7, there is shown a high-level block diagram of an exemplary computer system 701 that may be used to implement one or more of the methods, tools, and modules (e.g., using one or more processor circuits or computer processors of a computer) described herein, as well as any related functionality, in accordance with embodiments of the present disclosure. In some embodiments, the main components of computer system 701 may include one or more CPUs 702, a memory subsystem 704, a terminal interface 712, a storage interface 716, an I / O (input / output) device interface 714, and a network interface 718, all of which may be communicatively coupled, directly or indirectly, via a memory bus 703, an I / O bus 708, and an I / O bus interface 710 for component-to-component communication.

[0101] Computer system 701 may include one or more general-purpose programmable central processing units (CPUs) 702A, 702B, 702C, and 702D, collectively referred to herein as CPU 702. In some embodiments, computer system 701 may include multiple processors typical of a relatively large system, while in other embodiments, computer system 701 may alternatively be a single-CPU system. Each CPU 702 may execute instructions stored in memory subsystem 704 and may include one or more levels of on-board cache. In some embodiments, the processor may include at least one or more of a memory controller and / or a storage controller. In some embodiments, the CPU may execute processes included herein (e.g., process 600 described in FIG. 6). In some embodiments, computer system 701 may be configured as the object deformation analysis system 100 of FIG. 1.

[0102] The system memory subsystem 704 can include a computer system readable medium in the form of volatile memory such as random access memory (RAM) 722 or cache memory 724. The computer system 701 can further include other removable / non-removable, volatile / non-volatile computer system data storage media. By way of example only, the storage system 726 can be provided for reading from and writing to a non-removable non-volatile magnetic medium such as a "hard drive". Although not shown, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), or an optical disk drive for reading from and writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media can be provided. Further, the memory subsystem 704 can include flash memory, e.g., a flash memory stick drive or a flash drive. The memory devices can be connected to the memory bus 703 by one or more data media interfaces. The memory subsystem 704 can include at least one program product having a set (e.g., at least one) of program modules configured to execute the functions of various embodiments.

[0103] Memory bus 703 is shown in FIG. 7 as a single bus structure that provides a direct communication path between CPU 702, memory subsystem 704, and I / O bus interface 710. However, in some embodiments, memory bus 703 may include a plurality of different buses or communication paths, which may be arranged in any of a variety of forms, such as point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, parallel and redundant paths, or any other suitable type of configuration. Further, although I / O bus interface 710 and I / O bus 708 are shown as a single unit, computer system 701 may include, in some embodiments, a plurality of I / O bus interfaces 710, a plurality of I / O buses 708, or both. Additionally, while a plurality of I / O interface units are shown that separate I / O bus 708 from various communication paths extending to various I / O devices, in other embodiments, some or all of the I / O devices may be directly connected to one or more system I / O buses.

[0104] In some embodiments, computer system 701 may be a multi-user mainframe computer system, a single-user system, or a server computer or similar device that has little or no direct user interface but receives requests from other computer systems (clients). Further, in some embodiments, computer system 701 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, a network switch or router, or any other suitable type of electronic device.

[0105] Note that FIG. 7 is intended to show representative major components of an exemplary computer system 701. However, in some embodiments, individual components may have higher or lower complexity than those shown in FIG. 7, there may be components other than those shown in FIG. 7, or additional components, and the number, type, and configuration of such components may vary.

[0106] One or more programs / utilities 728, each having a set 730 of at least one program module, may be stored in the memory subsystem 704. The programs / utilities 728 may include a hypervisor (also called a virtual machine monitor), one or more operating systems, one or more application programs, other program modules, and program data. Each of these operating systems, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation of a networking environment. The programs / utilities 728 and / or the program modules 730 generally execute the functions or methodologies of the various embodiments.

[0107] Various aspects of the present disclosure are described by the machine logic block diagrams included in the description, flowcharts, block diagrams of computer systems, and / or embodiments of a computer program product (CPP). With respect to any flowchart, depending on the relevant technology, operations can be performed in an order different from that shown in a given flowchart. For example, again depending on the relevant technology, two operations shown in consecutive flowchart blocks can be performed in reverse order, as a single integrated step, simultaneously, or in a manner that is at least partially temporally overlapping.

[0108] An embodiment of a computer program product (referred to herein as a "CPP embodiment" or "CPP") is, in the present disclosure, a set of one or more storage devices that collectively include machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. The term "storage device" is any tangible device that can hold and store instructions for use by a computer processor. By way of non-limiting example, a computer-readable storage medium may be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media are floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pit lands formed on the major surface of a disk), or any suitable combination of the foregoing. The term computer-readable storage medium as used in the present disclosure should not be construed to include storage in the form of its own transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses passing through an optical fiber cable, electrical signals communicated by wire, and / or other transmission media. As will be understood by those skilled in the art, data typically moves intermittently during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the storage device is not transient because the data is not transient while it is being stored.

[0109] As will be described in more detail herein, it is contemplated that some or all of the operations of some embodiments of the methods described herein may be performed in alternative orders or not at all. Further, multiple operations may occur simultaneously or within a larger process.

[0110] Embodiments of the present disclosure may be implemented with virtually any type of computer, regardless of whether the platform is suitable for storing and / or executing program code. FIG. 8 shows, by way of example, a computing environment 800 (e.g., a cloud computing system) suitable for executing program code related to the methods disclosed herein and for object deformation analysis and management. In some embodiments, the computing environment 800 may be the same as or an implementation of the computing environment 100.

[0111] Computing environment 800 includes an example of an environment for executing at least a portion of computer code related to the execution of the method of the invention, such as object deformation analysis code 900. Object deformation analysis code 900 may be an implementation of the codebase of autonomous vehicle management system 100. In addition to object deformation analysis code 900, computing environment 800 includes, for example, computer 801, wide area network (WAN) 802, end user device (EUD) 803, remote server 804, public cloud 805, and private cloud 806. In this embodiment, computer 801 includes a processor set 810 (including processing circuit 820 and cache 821), communication fabric 811, volatile memory 812, persistent storage 813 (including operating system 822 and object deformation analysis code 900 as identified above), a set of peripheral devices 814 (including a set of user interface (UI) devices 823, storage 824, and a set of Internet of Things (IoT) sensors 825), and network module 815. Remote server 804 includes remote database 830. Public cloud 805 includes gateway 840, cloud orchestration module 841, a set of host physical machines 842, a set of virtual machines 843, and a set of containers 844.

[0112] Computer 801 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch, or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device currently known or developed in the future that can execute programs, access networks, or query databases such as remote database 830. As is well understood in the technical field of computer technology and as appropriate for the technology, the execution of computer-implemented methods can be distributed among multiple computers and / or among multiple locations. On the other hand, in the present presentation of computing environment 800, the detailed discussion focuses on a single computer, specifically computer 801, to make the presentation as simple as possible. Computer 801 can be located in the cloud but is not shown in the cloud in FIG. 1. On the other hand, computer 801 does not need to be in the cloud, except where it can be affirmatively shown.

[0113] Processor set 810 includes one or more computer processors of any type currently known or developed in the future. Processing circuitry 820 can be distributed across multiple packages, such as multiple cooperative integrated circuit chips. Processing circuitry 820 can implement multiple processor threads and / or multiple processor cores. Cache 821 is memory located in the processor chip package and used for data or code that needs to be available for rapid access by threads or cores typically executing on processor set 810. Cache memory is usually organized into multiple levels according to its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set can be located "off-chip". In some computing environments, processor set 810 can be designed to cooperate with qubits to perform quantum computing.

[0114] Computer-readable program instructions are typically loaded into computer 801 and cause a set of operational steps to be executed by processor set 810 of computer 801, thereby implementing a computer-implemented method. As a result, the instructions so executed instantiate the method specified in the flowchart and / or narrative description of the computer-implemented method (collectively referred to herein as "the method of the invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 821 and other storage media described below. The program instructions and associated data are accessed by processor set 810 to control and direct the execution of the method of the invention. In computing environment 800, at least a portion of the instructions for executing the method of the invention may be stored in object deformation analysis code 900 in persistent storage 813.

[0115] Communication fabric 811 is a signal transmission path that enables various components of computer 801 to communicate with each other. Typically, this fabric consists of switches and conductive paths, such as buses, bridges, physical input / output ports, and switches and conductive paths that make up the like. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0116] Volatile memory 812 is any type of volatile memory known currently or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 812 is characterized by random access, although this is not necessary unless affirmatively indicated. In computer 801, volatile memory 812 is located within a single package and inside computer 801. Alternatively or additionally, volatile memory may be distributed across multiple packages and / or located outside computer 801.

[0117] The persistent storage 813 is any form of non-volatile storage for computers that are currently known or will be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is directly supplied to the computer 801 and / or to the persistent storage 813. The persistent storage 813 can be read-only memory (ROM), but usually at least a part of the persistent storage enables writing of data, deletion of data, and re-writing of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 822 can take a plurality of forms such as various known proprietary operating systems or an open-source portable operating system interface type operating system that employs a kernel. The code included in the object deformation analysis code 900 usually includes at least a part of the computer code involved in the execution of the method of the invention.

[0118] The peripheral device set 814 includes a set of peripheral devices of the computer 801. The data communication connections between the peripheral devices and other components of the computer 801 can be implemented in various ways, such as a Bluetooth (registered trademark) connection, a Near Field Communication (NFC) connection, a connection by a cable (such as a Universal Serial Bus (USB) type cable), an insertion type connection (for example, a Secure Digital (SD) card), a connection by a local area communication network, and even a connection through a wide area network such as the Internet. In various embodiments, the UI device set 823 may include components such as a display screen, a speaker, a microphone, wearable devices (such as goggles and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a tactile device. The storage 824 is an external storage such as an external hard drive or an insertable storage such as an SD card. The storage 824 can be persistent and / or volatile. In some embodiments, the storage 824 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 801 is required to have a large amount of storage (for example, the computer 801 locally stores and manages a large database), this storage can be provided by a peripheral storage device designed to store a very large amount of data, such as a Storage Area Network (SAN) shared by a plurality of geographically dispersed computers. The IoT sensor set 825 is composed of sensors that can be used in Internet of Things applications. For example, one sensor can be a thermometer, and another sensor can be a motion detector.

[0119] The network module 815 is a collection of computer software, hardware, and firmware that enables computer 801 to communicate with other computers through WAN 802. The network module 815 may include hardware such as a modem or Wi-Fi (registered trademark) signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control function and network transfer function of the network module 815 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control function and transfer function of the network module 815 are executed on physically separated devices, whereby the control function manages multiple different network hardware devices. The computer-readable program instructions for executing the inventive method may typically be downloaded to computer 801 from an external computer or external storage device through a network adapter card or network interface included in the network module 815.

[0120] WAN 802 is any wide area network (e.g., the Internet) capable of communicating computer data at non-local distances by any technology for communicating computer data known currently or developed in the future. In some embodiments, WAN 802 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area such as a Wi-Fi (registered trademark) network. A WAN and / or a LAN typically includes computer hardware such as copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0121] The end - user device (EUD) 803 is any computer system used and controlled by an end - user (e.g., a customer of the enterprise operating the computer 801) and can take any form described above in relation to the computer 801. The EUD 803 typically receives useful data from the operation of the computer 801. For example, in a virtual case where the computer 801 is designed to provide recommendations to the end - user, this recommendation is typically communicated from the network module 815 of the computer 801 to the EUD 803 through the WAN 802. Thus, the EUD 803 can display the recommendation to the end - user or, if not, present it. In some embodiments, the EUD 803 can be a client device such as a thin - client, a thick - client, a mainframe computer, and a desktop computer.

[0122] The remote server 804 is any computer system that provides at least some data and / or functions to the computer 801. The remote server 804 can be controlled and used by the same entity that operates the computer 801. The remote server 804 represents a machine that collects and stores useful data that is useful for use by other computers such as the computer 801. For example, in a virtual case where the computer 801 is designed and programmed to provide recommendations based on past data, this past data can be provided from the remote database 830 of the remote server 804 to the computer 801.

[0123] The public cloud 805 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without direct and active management by the user. Cloud computing typically exploits resource sharing to achieve coherence and economies of scale. The direct and active management of the computing resources of the public cloud 805 is performed by the computer hardware and / or software of the cloud orchestration module 841. The computing resources provided by the public cloud 805 are typically implemented by a virtual computing environment that runs on various computers that are part of the physical computers in the public cloud 805 and / or the host physical machine set 842 available thereto. The virtual computing environment (VCE) typically takes the form of virtual machines from a virtual machine set 843 and / or containers from a container set 844. It is understood that these VCEs can be saved as images and transferred among and between various physical machine hosts, either as images or after instantiation of the VCE. The cloud orchestration module 841 manages the transfer and storage of images, deploys new instantiations of the VCE, and manages the active instantiation of VCE deployments. The gateway 840 is a collection of computer software, hardware, and firmware that enables the public cloud 805 to communicate through the WAN 802.

[0124] Some further explanation of a virtualized computing environment (VCE) is provided here. A VCE can be saved as an "image". A new active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of the operating system where the kernel enables the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave as actual computers from the perspective of the programs running within them. A computer program running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices allocated to the container. This is a function known as containerization.

[0125] The private cloud 806 is similar to the public cloud 805, except that computing resources are available for use by only a single enterprise. The private cloud 806 is depicted as communicating with the WAN 802, but in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composition of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate discrete entity, but the larger hybrid cloud architecture is coupled by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability among the multiple constituent clouds. In this embodiment, the public cloud 805 and the private cloud 806 are both parts of a larger hybrid cloud.

[0126] This disclosure includes a detailed description of cloud computing, but it is understood that the implementations of the teachings described herein are not limited to cloud computing environments. Rather, embodiments of the present disclosure can be implemented in conjunction with any other type of computing environment, now known or later developed. In some embodiments, one or more of the operating system 822 and the object transformation analysis code 900 can be implemented as a service model. The service model can include software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). In SaaS, the capabilities provided to the consumer are to use the provider's applications that run on the cloud infrastructure. The applications are accessible from various client devices via a client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even the individual application capabilities, except for limited user-specific application configuration settings in some cases. In PaaS, the capabilities provided to the consumer are to deploy the applications created or obtained by the consumer using the programming languages and tools supported by the provider onto the cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but controls the deployed applications and, in some cases, the configuration of the application hosting environment. In IaaS, the capabilities provided to the consumer are to provision processing, storage, network, and other basic computing resources, and the consumer can deploy and run any software that can include an operating system and applications.Consumers do not manage or control the underlying cloud infrastructure, but control the operating system, storage, deployed applications, and in some cases, limitedly control selected networking components (e.g., host firewalls).

[0127] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0128] These computer readable program instructions may be provided to the processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium containing the instructions comprises a manufacture including instructions which implement the function / act specified in one or more blocks of the flowchart and / or block diagram.

[0129] Also, computer-readable program instructions, when executed on a computer, other programmable data processing apparatus, or other devices, cause the functions / operations specified in one or more blocks of a flowchart and / or block diagram to be implemented, are loaded into the computer, other programmable data processing apparatus, or other devices, and cause a series of operational steps to be executed on the computer, other programmable data processing apparatus, or other devices, thereby generating a computer-implemented process.

[0130] The flowcharts and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions that include one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions described in the blocks may be performed in an order different from that depicted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or a combination of dedicated hardware and computer instructions.

[0131] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly dictates otherwise. The terms “comprise” and / or “comprising,” as used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0132] All means or steps and corresponding structures, materials, acts, and equivalents of the functional elements in the following claims are intended to include any structure, material, or act for performing functions in combination with elements of other claims as specifically claimed. Although the description of the disclosure has been presented for purposes of illustration and description, it is not intended to be exhaustive or to limit the disclosure to the forms disclosed. Without departing from the scope of the disclosure, many modifications and variations will be apparent to those skilled in the art. Embodiments are chosen and described in order to explain the principles of the disclosure and its practical application, and to enable others skilled in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular intended use.

[0133] The description of the various embodiments of the present disclosure is presented for purposes of illustration and is not intended to be exhaustive or to limit the disclosure to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to explain the principles of the embodiments, actual applications, or technical improvements over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. identifying a plurality of objects in the first stack configuration based on an analysis of the image data; determining a load distribution for each object of the plurality of objects relative to a subset of objects of the plurality of objects in the first stacked configuration; generating a visualization depicting a deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects in the first stacked configuration based on the load distribution of each object using a generative adversarial network (GAN) algorithm; and displaying to a user said visualization depicting said deformation of each object of said plurality of objects. A computer-implemented method comprising:

2. determining a time period during which the plurality of objects are in the first stacked configuration; generating a set of predicted deformation patterns for each of the plurality of objects over a time period based on the load distribution for each object using the GAN algorithm; and displaying to the user the sequence of predicted deformation patterns for each object of the plurality of objects over the time period. The computer-implemented method of claim 1 , further comprising:

3. determining a second stack configuration based on the identified plurality of objects and the load distribution for each object in the first stack configuration, the second stack configuration minimizing the deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects; generating a visualization of the minimized deformation of each object of the plurality of objects in the second stacked configuration using the GAN algorithm; and displaying the visualization of the plurality of objects in the second stacked configuration to the user. The computer-implemented method of claim 1 , further comprising:

4. identifying, based on an analysis of second image data, that a new object has been added by the user to the plurality of objects organized in the second stack configuration; evaluating an impact of the new object on the load distribution for each object of the plurality of objects in the second stacked configuration; determining a location for placing the new object in the second stack configuration, where the location for placing the new object minimizes the deformation of each object relative to the subset of objects in the second stack configuration; generating a visualization of the position for placing the new object in the second stack configuration using the GAN algorithm; and displaying the visualization of the location for placing the new object within the second stack configuration. The computer-implemented method of claim 3 , further comprising:

5. The computer-implemented method of claim 1 , wherein generating the visualization depicting deformations of each object of the plurality of objects comprises analyzing packaging specifications of one or more objects of the plurality of objects.

6. 2. The computer-implemented method of claim 1, wherein determining the load distribution for each object of the plurality of objects is based on a set of object attributes associated with product identification codes identified via the analysis of the image data.

7. 2. The computer-implemented method of claim 1, wherein determining the load distribution for each object of the plurality of objects for a subset of objects of the plurality of objects in the first stacked configuration comprises determining a dynamic load distribution based on a movement applied to the first stacked configuration.

8. The computer-implemented method of claim 1 , wherein the image data is generated by an Internet of Things (IoT) camera.

9. The computer-implemented method of claim 1 , wherein the visualization depicting the deformation of each object of the plurality of objects is displayed to the user via augmented reality glasses.

10. A processor; and A computer-readable storage medium communicatively coupled to the processor and storing program instructions. Equipped with The program instructions, when executed by the processor, cause the processor to: identifying a plurality of objects in the first stack configuration based on an analysis of the image data; determining a load distribution for each of the plurality of objects relative to a subset of objects of the plurality of objects in the first stacked configuration; generating a visualization depicting a deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects in the first stacked configuration based on the load distribution for each object using a generative adversarial network (GAN) algorithm; and displaying to a user said visualization depicting said deformation of each object of said plurality of objects. Executing a method comprising: system.

11. determining a time period during which the plurality of objects are in the first stacked configuration; generating a set of predicted deformation patterns for each of the plurality of objects over a time span based on the load distribution for each object using the GAN algorithm; and displaying to the user the sequence of predicted deformation patterns for each object of the plurality of objects over the time period. The system of claim 10 further comprising:

12. determining a second stacking configuration that minimizes the deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stacking configuration; generating a visualization of the minimized deformation of each object of the plurality of objects in the second stacked configuration using the GAN algorithm; and displaying the visualization of the plurality of objects in the second stacked configuration to the user. The system of claim 10 further comprising:

13. identifying, based on an analysis of second image data, that a new object has been added by the user to the plurality of objects organized in the second stack; evaluating an impact of the new object on the load distribution for each object of the plurality of objects in the second stacked configuration; determining a location for placing the new object in the second stack configuration, where the location for placing the new object minimizes the deformation of each object relative to the subset of objects in the second stack configuration; generating a visualization of the position for placing the new object in the second stack configuration using the GAN algorithm; and displaying the visualization of the position for placing the new object within the second stack configuration. The system of claim 12 further comprising:

14. The system of claim 10 , wherein generating the visualization depicting deformation of each object of the plurality of objects comprises analyzing packaging specifications of one or more objects of the plurality of objects.

15. 15. The system of claim 10, wherein determining the load distribution for each object of the plurality of objects is based on a set of object attributes associated with product identification codes identified via the analysis of the image data.

16. 15. The system of claim 10, wherein the step of determining the load distribution for each object of the plurality of objects for a subset of objects of the plurality of objects in the first stack configuration comprises a step of determining a dynamic load distribution based on a movement applied to the first stack configuration.

17. A computer program having program instructions embodied therein, the program instructions being executable by a processor and causing the processor to: identifying a plurality of objects in the first stack configuration based on an analysis of the image data; determining a load distribution for each of the plurality of objects relative to a subset of objects of the plurality of objects in the first stacked configuration; generating a visualization depicting a deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects in the first stacked configuration based on the load distribution for each object using a generative adversarial network (GAN) algorithm; and displaying to a user said visualization depicting said deformation of each object of said plurality of objects. A computer program product for causing a computer to carry out a method comprising:

18. determining a time period during which the plurality of objects are in the first stacked configuration; generating a set of predicted deformation patterns for each of the plurality of objects over a time span based on the load distribution for each object using the GAN algorithm; and displaying to the user the sequence of predicted deformation patterns for each object of the plurality of objects over the time period.

20. The computer program product of claim 17, further comprising:

19. determining a second stacking configuration that minimizes the deformation of each object of the plurality of objects relative to the subset of objects of the plurality of objects based on the identified plurality of objects and the load distribution for each object in the first stacking configuration; generating a visualization of the minimized deformation of each object of the plurality of objects in the second stacked configuration using the GAN algorithm; and displaying the visualization of the plurality of objects in the second stacked configuration to the user.

19. The computer program of claim 17 or 18, further comprising:

20. identifying, based on an analysis of second image data, that a new object has been added by the user to the plurality of objects organized in the second stack; evaluating an impact of the new object on the load distribution for each object of the plurality of objects in the second stacked configuration; determining a location for placing the new object in the second stack configuration, where the location for placing the new object minimizes the deformation of each object relative to the subset of objects in the second stack configuration; generating a visualization of the position for placing the new object in the second stack configuration using the GAN algorithm; and displaying the visualization of the position for placing the new object within the second stack configuration.

20. The computer program product of claim 19, further comprising:

Images