Integrated waste management system and method therefor

Abstract

Provided are an integrated waste management system and a method therefor. The integrated waste management system according to some embodiments may comprise: a food waste collection apparatus that generates a food waste image of a user by photographing one or more dishes containing leftovers, acquires weight information by measuring the weight of the one or more dishes through a weight sensor, and provides food waste discharge records comprising the food waste image and the weight information; and an integrated waste management server that integrally manages waste discharge information of the user on the basis of the food waste discharge records. According to the system, a sophisticated reward system can be constructed by integrally managing waste discharge information for each user, and as a result, waste discharge amount reduction and the formation of desirable waste discharge habits can be effectively induced.

Description

Integrated waste management system and method thereof

[0001] The present disclosure relates to a technology for integrating and managing discharge information of various types of waste, such as food waste and recyclable waste, on an individual basis.

[0002] With the development of a capitalist society characterized by mass production and mass consumption, the generation of various types of household waste, including food waste and recyclable waste, is steadily increasing. While it is desirable to properly handle this household waste to ensure resource recycling and prevent environmental pollution, the reality is that treatment efficiency is low due to the hassle of separating waste, inadequate collection infrastructure, and the absence of a reward system for users.

[0003] In particular, many users often mix different types of waste or dispose of them inappropriately, resulting in deteriorating waste quality and increased disposal costs. For example, food waste is often discharged mixed with foreign substances, resulting in low rates of animal feed and resource recovery. Even for recyclable waste, its quality or mixed discharge often hinders practical utilization.

[0004] To address the problems of quality degradation and increased disposal costs caused by such waste disposal practices, measures are needed to encourage individual users to reduce their waste output and develop desirable disposal habits. Furthermore, to achieve this, a technological foundation must be established to collect and integrate waste disposal information at the user (i.e., individual) level.

[0005] A technical problem to be solved through several embodiments of the present disclosure is to provide a system and method for integrated management of discharge information of various types of waste for each user.

[0006] Another technical problem to be solved by some embodiments of the present disclosure is to provide a system and method that can encourage each user to reduce waste discharge and form desirable waste disposal habits.

[0007] The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art of the present disclosure from the description below.

[0008] An integrated waste management system according to some embodiments of the present disclosure for solving the above-described technical problem may include a food waste collection device that takes a picture of one or more dishes containing leftover food to create a food waste image of a user, measures the weight of the one or more dishes through a weight sensor to obtain weight information, and provides a food waste discharge record including the food waste image and the weight information; and an integrated waste management server that comprehensively manages the user's waste discharge information based on the food waste discharge record.

[0009] In some embodiments, the food waste collection device receives a specific order number randomly issued by a food ordering device from the user, transmits the specific order number to the integrated waste management server, and the integrated waste management server receives order numbers and identification information matching the order numbers from the food ordering device, and obtains identification information of the user matching the specific order number from the identification information, wherein the identification information may be information entered by the user into the food ordering device during the order information input process.

[0010] In some embodiments, the device further includes a recycling waste collection device that takes a picture of the recycling waste input by the user to create an image of the recycling waste and provides a recycling waste discharge record including the image of the recycling waste, and the integrated waste management server can manage the waste discharge information further based on the recycling waste discharge record.

[0011] In some embodiments, the integrated waste management server calculates the user's first discharge amount for a food waste item based on the food waste discharge record, compares the first discharge amount with a first average discharge amount of a plurality of users for the food waste item to determine the user's first environmental contribution, analyzes the recyclable waste discharge record to calculate the user's discharge quality score, determines the user's second environmental contribution for a recyclable waste item based on the discharge quality score, determines the user's integrated environmental contribution by synthesizing the first environmental contribution and the second environmental contribution based on a weighting, wherein a weighting given to the first environmental contribution is higher than the second environmental contribution, and calculates a reward to be paid to the user based on the integrated environmental contribution.

[0012] In some embodiments, the integrated waste management server may detect one or more foreign matter objects in the food waste image through a first deep learning model, adjust the first environmental contribution downward based on the detection result, analyze the recyclable waste image through a second deep learning model to determine whether foreign matter is included, the degree of contamination, whether packaging has been removed, and the accuracy of separation and discharge, thereby calculating the discharge quality score, and determine the integrated environmental contribution by synthesizing the adjusted first environmental contribution and the second environmental contribution.

[0013] In some embodiments, the integrated waste management server may evaluate the level of emission improvement by comparing the current emission amount derived from the food waste emission record with the past emission amount of the user, evaluate the level of food waste by comparing the amount of food provided to the user with the current emission amount, and adjust the first environmental contribution based on the level of emission improvement and the level of food waste.

[0014] In some embodiments, the integrated waste management server analyzes the food waste discharge record to derive information about one or more food items included in the leftovers and the amount of discharge of each food item, calculates a discharge burden score for the food waste based on the amount of discharge of each food item and an item weight, and adjusts the first environmental contribution based on the discharge burden score, wherein the item weight may be determined based on the carbon emissions of each food item.

[0015] In some embodiments, the food waste discharge record further includes food order information of the user, and the food waste discharge record is analyzed through a deep learning model, wherein the deep learning model may include: an image encoder that encodes the food waste image to generate a first latent representation; an order encoder that encodes the food order information to generate a second latent representation; an aggregator that aggregates the first latent representation and the second latent representation to generate an integrated representation; a classification head that receives the integrated representation and the weight information as input and predicts the type of the one or more food items; and a regression head that receives the integrated representation and the weight information as input and predicts the discharge amount of each food item.

[0016] In some embodiments, the integrated waste management server detects one or more dish objects in the food waste image, subtracts the weight of the detected dish objects from the weight information to calculate a net weight of the food waste, trains a deep learning model using the net weight as a label for the food waste image, and predicts the net weight of the food waste included in a subsequent food waste image using the trained deep learning model.

[0017] In some embodiments, the integrated waste management server obtains unit discharge information corresponding to the current discharge behavior of the user from the food waste discharge record, determines whether the current discharge behavior is an abnormal discharge behavior based on the unit discharge information, and if it is determined to be an abnormal discharge behavior: obtains waste discharge history corresponding to past discharge behaviors of the user, analyzes the unit discharge information and the waste discharge history through a deep learning model to estimate a possibility that the current discharge behavior is an intentional discharge behavior, and imposes a penalty on the user if the estimated possibility is greater than a reference value, wherein the deep learning model may include: a sequence encoder encoding the unit discharge information and the waste discharge history; and a predictor estimating the possibility based on the encoding result.

[0018] In some embodiments, the integrated waste management server calculates the user's environmental contribution by applying a weight to the food waste discharge record when the food waste discharge record satisfies a preset condition, wherein the preset condition may include a case where the food waste discharge record was generated from the food waste collection device installed at a specific location and a case where the food waste discharge record was generated during a specific event period.

[0019] According to some embodiments of the present disclosure for solving the above-described technical problem, an integrated waste management method is provided, which is performed in a system including a food waste collection device and an integrated waste management server, comprising: a step of collecting a user's food waste discharge record from the food waste collection device; and a step of comprehensively managing the user's waste discharge information based on the food waste discharge record, wherein the food waste collection device includes a weight sensor and a camera sensor, and captures a dish containing leftover food through the camera sensor to create a food waste image, measures the weight of the dish through the weight sensor to obtain weight information, and transmits the food waste discharge record including the food waste image and the weight information to the integrated waste management server.

[0020] According to some embodiments of the present disclosure for solving the above-described technical problem, a computer program may be stored in a computer-readable recording medium to execute the steps of collecting a user's food waste discharge record from a food waste collection device; and comprehensively managing the user's waste discharge information based on the food waste discharge record, in combination with a computer processor.

[0021] According to some embodiments of the present disclosure, each user's waste disposal records can be automatically collected through waste collection devices (e.g., food waste collection devices, recyclable waste collection devices, etc.) installed at various sites. In this case, information on various types of waste generated at various locations can be accurately and integratedly managed on a user-by-user basis, and based on this, a sophisticated reward system that provides differentiated rewards based on environmental contribution can be effectively established.

[0022] In addition, by calculating the integrated environmental contribution of various types of waste based on the collected waste discharge records (or waste discharge information) and rewarding or imposing penalties on users based on this, users can be effectively encouraged to reduce their waste discharge and develop desirable discharge habits.

[0023] Additionally, by comparing the emissions of individual users with the average emissions of all users, the environmental contribution of individual users to food waste items can be accurately assessed.

[0024] Additionally, based on the discharge quality score for recyclable waste, the environmental contribution of individual users for recyclable waste items can be accurately assessed.

[0025] Additionally, by performing online learning on a deep learning model (e.g., a model used for waste image analysis) using waste images collected in real-time waste discharge records, the performance of the deep learning model can be gradually improved over time.

[0026] The effects according to the technical idea of ​​the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0027] FIG. 1 is a diagram illustrating an exemplary operating environment of an integrated waste management system according to some embodiments of the present disclosure.

[0028] FIG. 2 is an exemplary flowchart schematically illustrating an integrated waste management method according to some embodiments of the present disclosure.

[0029] FIG. 3 is an exemplary diagram illustrating a process for collecting waste discharge records according to some embodiments of the present disclosure.

[0030] FIGS. 4 and 5 are exemplary flowcharts illustrating a method for calculating environmental contribution according to some embodiments of the present disclosure.

[0031] FIG. 6 is an exemplary flowchart illustrating a method for calculating environmental contribution according to some other embodiments of the present disclosure.

[0032] FIGS. 7 and 8 are exemplary drawings illustrating a deep learning model for predicting food waste weight according to some embodiments of the present disclosure.

[0033] FIG. 9 is an exemplary diagram illustrating a deep learning model for food waste image analysis according to some embodiments of the present disclosure.

[0034] FIG. 10 is an exemplary diagram illustrating a deep learning model for image analysis of recyclable waste according to some embodiments of the present disclosure.

[0035] FIG. 11 and FIG. 12 are exemplary drawings illustrating a deep learning model for determining intentional discharge behavior according to some embodiments of the present disclosure.

[0036] FIG. 13 illustrates an exemplary computing device that can implement an integrated waste management server or the like according to some embodiments of the present disclosure.

[0037] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure, and methods for achieving them, will become clear with reference to the embodiments described in detail below together with the attached drawings. However, the technical idea of ​​the present disclosure is not limited to the following embodiments and may be implemented in various different forms. The following embodiments are provided only to complete the technical idea of ​​the present disclosure and to fully inform those skilled in the art of the present disclosure of the scope of the present disclosure, and the technical idea of ​​the present disclosure is defined only by the scope of the claims.

[0038] In describing various embodiments of the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description will be omitted.

[0039] Unless otherwise defined, the terms (including technical and scientific terms) used in the following examples may be used with meanings commonly understood by those of ordinary skill in the art to which this disclosure pertains; however, this may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. The terminology used in this disclosure is for the purpose of describing the embodiments and is not intended to limit the scope of this disclosure.

[0040] In the following examples, singular expressions include plural concepts unless the context clearly specifies that they are singular. Furthermore, plural expressions include singular concepts unless the context clearly specifies that they are plural.

[0041] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are only used to distinguish certain components from other components, and the nature, order, or sequence of the components are not limited by the terms.

[0042] In the following embodiments, components described using terms such as ~part or unit, module, block, ~or, ~er, etc. and functional blocks illustrated in the drawings may be implemented in the form of software, hardware, or a combination thereof. Software may include, for example, machine code, firmware, embedded code (or software), application software, or a combination thereof. In addition, hardware may include, for example, electric circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, passive components, or a combination thereof. As more specific examples, for example, ~part, module, etc. may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0043] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0044] FIG. 1 is a diagram illustrating an exemplary operating environment of an integrated waste management system according to some embodiments of the present disclosure.

[0045] As illustrated in FIG. 1, the integrated waste management system according to the embodiments may be configured to include an integrated waste management server (10) and a plurality of waste collection devices (11-1 to 11-N), and may operate in conjunction with a plurality of user terminals (12-1 to 12-K).

[0046] Hereinafter, for the convenience of understanding, the reference number '11' will be used both when collectively referring to a plurality of waste collection devices (11-1 to 11-N) or when referring to any waste collection device (e.g., 11-1 or 11-2). Similarly, the reference number '12' will be used both when collectively referring to a plurality of user terminals (12-1 to 12-K) or when referring to any user terminal (e.g., 12-1 or 12-2). In addition, for the sake of brevity of explanation, the integrated waste management server (10) will be abbreviated as 'management server'.

[0047] A waste collection device (11) is a device that collects waste discharged by a user, creates a discharge record (i.e., data) thereof, and provides it to a management server (10). Examples of such waste collection devices (11) include, but are not limited to, food waste collection devices and recyclable waste collection devices.

[0048] The waste collection device (11) may be configured to be equipped with various types of sensors (e.g., camera sensors, weight sensors, etc.) and to generate a waste discharge record (hereinafter, may be abbreviated as “discharge record”) of each user through these. The waste discharge record may include detailed information such as, but not limited to, a discharge location, discharge date and time, a waste image, identification information of the discharge user, waste weight (i.e., weight), order information (e.g., food order information, etc.). In addition, the waste collection device (11) may transmit the waste discharge record of each user to the management server (10). At this time, the waste collection device (11) may encrypt and transmit the waste discharge record to protect personal information.

[0049] For example, a food waste collection device (11) can capture images of one or more dishes (e.g., plates, containers, spoons, etc.) containing food waste through a camera sensor to create a food waste image, measure the weight including the dishes through a weight sensor, and transmit a discharge record including the food waste image and weight information to a management server (10).

[0050] As another example, the recyclable waste collection device (11) may capture images of recyclable waste introduced into the internal space through a camera sensor, measure the weight of the waste through a weight sensor, and transmit a discharge record including the recyclable waste image and weight information to the management server (10). Alternatively, the recyclable waste collection device (11) may transmit a discharge record excluding weight information to the management server (10).

[0051] In some cases, a waste collection device (e.g., 11, food waste collection device, recyclable waste collection device, etc.) may analyze a waste image to derive status information of the waste (e.g., quantity, type, presence of foreign substances, presence of packaging removal, contamination level, etc.) and transmit a discharge record including the same to a management server (10). Such image analysis may be performed, for example, through a deep learning model (e.g., a lightweight neural network model, etc.), but the scope of the present disclosure is not limited thereto.

[0052] In addition, the waste collection device (11) can be installed in various sites such as restaurants, campgrounds, camping sites, local festival sites, neighborhood living facilities, residential areas, public facilities, etc., and can be utilized to collect waste discharge records of users. For example, a food waste collection device (11) installed in a restaurant can be utilized to collect food waste discharge records of users who visited the restaurant. As another example, a recyclable waste collection device (11) installed in a residential area can be utilized to collect recyclable waste discharge records of nearby users. As yet another example, a waste collection device (11) installed in a place with a specific purpose such as a campground, camping site, local festival site, etc. and / or a place operated during a specific event period can be utilized to collect waste discharge records limited to the place and / or event period.

[0053] In this way, waste discharge information for each user (i.e., individual) can be accurately collected and integrated through waste collection devices (11) installed at various sites, and a sophisticated reward system can be established that can induce reduction in waste discharge and formation of desirable discharge habits.

[0054] For detailed operation of the waste collection device (11), please refer to the description in Fig. 2, etc.

[0055] The waste collection device (11) may be named, in some cases, as a ‘waste collector’, ‘waste treatment device’, ‘waste treatment device / system’, ‘waste input device / system’, ‘waste discharge recording device / system’, etc.

[0056] The management server (10) is a computing device / system that comprehensively manages users' waste disposal information. The management server (10) collects users' waste disposal records through waste collection devices (11) installed at various sites, and based on this, manages waste disposal information for each user.

[0057] Waste discharge information may include, but is not limited to, discharge statistics / status information (e.g., total cumulative discharge amount, total average cumulative discharge amount, cumulative / average discharge amount by waste type, cumulative / average discharge amount by user, one-time average / cumulative discharge amount, daily / monthly average / cumulative discharge amount, cumulative / average discharge amount at a specific location, discharge amount change, etc.), discharge history, etc. In addition, the discharge history is composed of a sequence of unit discharge information, and unit discharge information may include, but is not limited to, discharge date and time, discharge location, discharge amount, waste image, waste status information, discharge quality score, discharge burden score, abnormal discharge type (e.g., excessive discharge, containing foreign substances, mixed discharge, unpackaged, etc.). Here, unit discharge information may be understood to mean discharge information corresponding to a specific unit (e.g., once, 1 day, 1 week, etc.), and may be composed including discharge records and / or analysis results derived therefrom.

[0058] Additionally, the management server (10) can calculate (or determine) each user's environmental contribution based on waste discharge records / information and provide rewards to the user based on their environmental contribution. Rewards may include, but are not limited to, points (e.g., carbon neutral points), mobile gift certificates, discount coupons, and public facility vouchers. Rewards can be designed in various ways without limitations on implementation form or method. A specific method for calculating a user's environmental contribution will be described below with reference to FIGS. 2 through 6.

[0059] In some cases, the management server (10) may impose penalties (e.g., fines, missions, warnings, etc.) on users for failing to meet environmental contribution standards. Penalties can also be designed in various ways, without limitations on implementation form or method.

[0060] In addition, the management server (10) can provide real-time or non-real-time information on waste discharge, environmental contribution, reward information, etc. for individual users or all users. For example, the management server (10) can provide users with notification messages containing discharge-related information (e.g., discharge amount, environmental contribution, discharge guidelines, etc.) in a periodic or non-periodical manner (e.g., whenever a discharge record is received). Such notification messages may be provided through a dedicated application installed on the user terminal (12), but the scope of the present disclosure is not limited thereto.

[0061] In addition, the management server (10) may be equipped with functions such as membership registration, login, and authentication for smooth management services, and may generate and provide a QR (Quick Response) code (or other identification code) for identifying individual users when registering as members. User identification information such as membership numbers may be recorded in the QR code.

[0062] The management server (10) may be named, in some cases, as a ‘central server / system’, an ‘integrated waste discharge management server / system’, a ‘waste reward server / system’, etc.

[0063] For detailed operation of the management server (10), please refer to the description of Fig. 2, etc.

[0064] The management server (10) described above may be implemented with at least one computing device. For example, all functions of the management server (10) may be implemented on a single computing device, or a first function of the management server (10) may be implemented on a first computing device and a second function may be implemented on a second computing device. Alternatively, specific functions of the management server (10) may be implemented on multiple computing devices.

[0065] A computing device may include any device equipped with computing capabilities; for an example of such a device, see FIG. 13. Since a computing device is a collection of interacting components (e.g., memory, processor, etc.), it may sometimes be referred to as a "computing system." Of course, the term "computing system" can also encompass the concept of a collection of interacting computing devices.

[0066] The user terminal (12) is a device used by users to check waste discharge information, contribution information, reward information, etc. A dedicated application linked to the management server (10) may be installed on the user terminal (12), and the user can view and / or confirm various information related to waste discharge or receive notification messages through the application. Alternatively, the user may access a webpage provided by the management server (10) via the terminal (12) and utilize the same service.

[0067] The user terminal (12) may be a mobile terminal such as a smartphone or laptop, or a fixed terminal such as a desktop. The user terminal (12) may be any type of device.

[0068] As illustrated, the waste collection device (11), the management server (10), and / or the user terminal (12) can communicate via a network. Here, the network can be implemented as any type of wired / wireless network, such as a local area network (LAN), a wide area network (WAN), a mobile radio communication network, Wibro (Wireless Broadband Internet), etc.

[0069] So far, with reference to FIG. 1, exemplary operating environments of integrated waste management systems according to some embodiments of the present disclosure have been described. Below, the operating method of the aforementioned integrated waste management system will be described in detail with reference to the drawings, including FIG. 2 and below.

[0070] Figure 2 is an exemplary flowchart illustrating an integrated waste management method according to some embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the objectives of the present disclosure, and it is understood that some steps may be added or deleted as needed.

[0071] FIG. 2 illustrates an example in which a management server (10) operates in conjunction with a food waste collection device (11-1) installed at a first site and a recyclable waste collection device (11-2) installed at a second site, but the scope of the present disclosure is not limited thereto, and the type and number of waste collection devices (11) may vary.

[0072] In addition, FIG. 2 illustrates an example in which a management server (10) collects waste disposal records of a specific user (i.e., an individual user who has registered as a member of the management server (10)) and provides rewards accordingly.

[0073] Hereinafter, with reference to the description of FIG. 2, the food waste collection device (11-1) and the recyclable waste collection device (11-2) are referred to as the ‘first collection device’ and the ‘second collection device’, respectively.

[0074] As illustrated in FIG. 2, the present embodiments may begin with steps S11 and S12, where the first collection device (11-1) generates a user's food waste discharge record and transmits it to the management server (10). For ease of understanding, steps S11 and S12 will be further described with reference to FIG. 3.

[0075] Figure 3 is an exemplary diagram illustrating the process of collecting a user's waste discharge record. Figure 3 illustrates a case where a first collection device (11-1) is installed in a waste disposal area within a restaurant (A, hereinafter referred to as "site 1") and a second collection device (11-2) is installed at a specific site (B, hereinafter referred to as "site 2").

[0076] As illustrated in FIG. 3, the first collection device (11-1) can detect a user's food waste discharge at the first site, create a discharge record thereof, and transmit the same to the management server (10). For example, the first collection device (11-1) can capture a photograph of tableware (e.g., a plate, a spoon, a container, etc.) containing leftover food using a camera sensor to create an image of food waste, and measure the weight including the tableware using a weight sensor to create a discharge record of the corresponding user. At this time, the discharge record can also include the user's identification information (e.g., membership number, name, personal phone number, etc.) (or information used to obtain identification information, such as an order number), food order information, etc.

[0077] The first collection device (11-1) can obtain the user's identification information in various ways. For example, the first collection device (11-1) can obtain the user's identification information in conjunction with the food ordering device (31). Specifically, the food ordering device (31) can receive the user's identification information (e.g., personal phone number, etc.) and food order information and issue an order number (e.g., the order number can be displayed on the printed receipt). The order number may be issued in a random manner, but the scope of the present disclosure is not limited thereto. In some cases, the order number may be issued sequentially. In addition, the first collection device (11-1) can receive the order number from the discharging user and, based on the order number, obtain the identification information matching the order number from the food ordering device (31). As another example, the first collection device (11-1) can obtain the user's identification information by recognizing a QR code displayed on the user terminal (e.g., a QR code issued upon membership registration). As another example, the first collection device (11-1) may transmit a discharge record including the user's order number to the management server (10), and the management server (10) may obtain the user's identification number matching the order number in conjunction with the food ordering device (31). In other words, the food ordering device (31) may transmit order numbers and matching identification information to the management server (10) in real time or near real time whenever it receives order numbers and matching identification information from individual users. Thereafter, when the first collection device (11-1) receives an order number (hereinafter referred to as a 'specific order number') from a specific user and transmits it to the management server (10), the management server (10) may obtain the identification information of a specific user matching the specific order number by comparing the specific order number with previously received order numbers. FIG. 3 illustrates an example in which the food ordering device (31) is implemented as a kiosk.

[0078] Meanwhile, in some cases, the first collection device (11-1) may analyze the food waste image using a deep learning model to derive waste status information including the net weight of the food waste, whether foreign substances are included, net weight (i.e., weight excluding food waste), etc., and transmit a discharge record including the information to the management server (10). For this, please refer to the descriptions of FIGS. 7 to 9, etc.

[0079] This is explained again with reference to Figure 2.

[0080] In step S13, the management server (10) processes the received (or collected) emission records. Here, processing of the emission records can be understood as including a task of calculating or updating the user's environmental contribution. For example, the management server (10) can update the user's waste emission information (e.g., accumulated amount of food waste discharged, etc.) and environmental contribution (e.g., accumulated contribution) based on the received emission records, and, if necessary, can transmit a notification message including the update result (e.g., current emission amount, accumulated emission amount, current contribution, accumulated contribution, etc.) to the user's terminal (12).

[0081] In this step S13, the management server (10) can perform various processing based on the received discharge records.

[0082] For example, the management server (10) can calculate the net weight of food waste by deducting the weight of the dishes from the weight information of the discharge record, and update the accumulated discharge amount of the corresponding user for the food waste item by reflecting the discharge amount corresponding to the net weight. If the type and number of dishes, etc. are fixed, the management server (10) can calculate the net weight by deducting the preset weight of the dishes. On the other hand, if the type and number of dishes, etc. are changed, the management server (10) can detect one or more dish objects in the food waste image and deduct the weight of the detected dish objects. The detection of the dish objects may be performed, for example, through a deep learning model that detects objects in an image (see FIG. 7), but the scope of the present disclosure is not limited thereto.

[0083] As another example, the management server (10) can analyze a food waste image to derive waste status information, including the type of food items contained in the waste, weight (i.e., weight), and whether foreign substances are present. This waste status information can be accurately derived using a deep learning model, and for this purpose, refer to the descriptions of FIGS. 7 to 9. The waste status information can be used, for example, to calculate environmental contribution, and for this purpose, refer to the descriptions of FIGS. 4 and 5.

[0084] As another example, the management server (10) can calculate a user's environmental contribution used in reward calculation. This will be described later with reference to FIGS. 4 and 5.

[0085] As another example, the management server (10) can continuously accumulate discharge records and use them to build deep learning models related to various tasks. For example, the management server (10) can build a deep learning model that predicts net weight from food waste images, a deep learning model that detects foreign substances, a deep learning model that predicts the types of food items contained in food waste and the amount of discharge (i.e., net weight), a deep learning model that determines intentional discharge acts (i.e., intentional abnormal discharge acts), etc. This will be described later with reference to FIGS. 7 to 12. In addition, the management server (10) can also perform online learning on the deep learning model using discharge records received in real time; for this, refer to the description of FIG. 8.

[0086] As another example, the management server (10) may perform processing based on various combinations of the examples described above.

[0087] In steps S14 and S15, the second collection device (11-2) creates a record of the user's recyclable waste discharge and transmits it to the management server (10). For better understanding, steps S14 and S15 will be further explained with reference to FIG. 3 again.

[0088] Referring back to FIG. 3, the second collection device (11-2) can detect the discharge of recyclable waste by a user at the second site, create a discharge record thereof, and transmit the same to the management server (10). For example, the second collection device (11-2) can capture images of recyclable waste discharged into the internal space using a camera sensor, and measure the weight of the recyclable waste using a weight sensor, thereby creating a discharge record of the user. At this time, the discharge record can also include the user's identification information (e.g., membership number, name, personal phone number, etc.) (or information used to obtain identification information).

[0089] The second collection device (11-2) can obtain user identification information in various ways. For example, the second collection device (11-2) can obtain user identification information by recognizing a QR code displayed on the user terminal (12) (e.g., a QR code issued upon membership registration). However, the scope of the present disclosure is not limited thereto. The second collection device (11-2) can also directly receive user identification information (e.g., a personal phone number, etc.).

[0090] Meanwhile, in some cases, the second collection device (11-2) may analyze images of recyclable waste using a deep learning model to derive waste status information including the number of recyclable waste, type, presence of foreign substances, removal of packaging (e.g., labels, etc.), contamination level, etc., and transmit a discharge record including the information to the management server (10). For this, please refer to the description of Fig. 10.

[0091] For reference, although FIG. 2 depicts step S13 (or step S14) as being performed after step S11 (or step S12), steps S11 and S13 may be performed regardless of the order.

[0092] This is explained again with reference to Figure 2.

[0093] In step S16, the management server (10) processes the received emission records. For example, the management server (10) can update the user's waste emission information (e.g., cumulative amount of recyclable waste) and environmental contribution (e.g., cumulative contribution) based on the received emission records, and, if necessary, can transmit a notification message including the update result (e.g., current emission amount, cumulative emission amount, emission quality score, current contribution, cumulative contribution, etc.) to the user's terminal (12). A specific method for calculating the environmental contribution of recyclable waste will be described later with reference to FIG. 6.

[0094] For this step S16, please refer to the description of step S13 described above.

[0095] In step S17, the management server (10) calculates an integrated environmental contribution based on the environmental contribution of each waste item. For example, the management server (10) can calculate an integrated environmental contribution by synthesizing (e.g., adding up, etc.) the environmental contributions of food waste items and recyclable waste items based on weights. At this time, the weights can be determined based on the processing cost, carbon emissions (e.g., carbon emissions during the processing process, carbon emissions during the production process, etc.), and pre-disposal price of each waste item. For example, since the carbon emissions, processing costs, etc. of food waste are higher than those of recyclable waste, a relatively higher weight can be assigned to the food waste item.

[0096] In step S18, the management server (10) may calculate rewards based on the integrated environmental contribution and provide the calculated rewards to the users. For example, the management server (10) may differentially calculate rewards based on the ranking of each user's integrated environmental contribution. In addition, the management server (10) may also determine penalties (e.g., fines, missions, warnings, etc.) to be imposed on users based on the integrated environmental contribution or its ranking. For example, if the integrated environmental contribution falls below a standard or is ranked low, the management server (10) may impose differential penalties on the users. Such rewards and penalties can be effectively utilized as a means to encourage reduction in waste discharge and the formation of proper disposal habits.

[0097] Heretofore, integrated waste management methods according to some embodiments of the present disclosure have been described with reference to FIGS. 2 and 3 . As described above, each user's waste disposal records can be automatically collected through waste collection devices (11-1, 11-2) installed at various sites. In this case, information on various types of waste disposal occurring at various locations can be accurately and integratedly managed on a user-by-user basis, and based on this, a sophisticated reward system that provides differentiated rewards based on environmental contribution can be effectively established.

[0098] In addition, by calculating the integrated environmental contribution of various types of waste based on the collected waste discharge records (or waste discharge information) and rewarding or imposing penalties on users based on this, users can be effectively encouraged to reduce their waste discharge and develop desirable discharge habits.

[0099] Hereinafter, various embodiments of a method for calculating a user's environmental contribution will be described with reference to FIGS. 4 to 6.

[0100] Figure 4 is an exemplary flowchart illustrating a method for calculating environmental contribution according to some embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the objectives of the present disclosure, and it is understood that some steps may be added or deleted as needed.

[0101] As illustrated in FIG. 4, the present embodiments relate to a method for calculating a user's environmental contribution (e.g., cumulative contribution, etc.) to a food waste item.

[0102] Specifically, the present embodiments may begin with step S41, which updates the user's cumulative amount of food waste for a food waste item by reflecting the user's food waste discharge record. For example, the management server (10) may calculate the net weight of food waste by subtracting the weight of the dishes from the weight information in the discharge record, and update the user's cumulative amount of food waste by reflecting the discharge amount corresponding to the net weight (i.e., the current discharge amount).

[0103] In step S42, the user's environmental contribution to food waste is determined by comparing the cumulative emissions with the overall average cumulative emissions. Specifically, the lower the user's cumulative emissions are compared to the overall average cumulative emissions, the higher the user's environmental contribution is calculated. This method of calculating environmental contribution can be understood as embodying the long-term reward design concept, which seeks to reward users who consistently engage in waste reduction activities.

[0104] For reference, the total average cumulative emissions refers to the average cumulative emissions of all users, and the management server (10) can calculate and continuously update the total average cumulative emissions based on the cumulative emissions of users registered as members.

[0105] In some cases, the management server (10) may compare the user's current discharge amount for a food waste item with the average discharge amount of all users (e.g., average discharge amount per time) to determine the environmental contribution, and may continuously and cumulatively manage such environmental contribution.

[0106] In step S43, the environmental contribution is adjusted according to preset rules. However, the specific method can be designed in various ways.

[0107] For example, the management server (10) can adjust the environmental contribution based on the difference between the user's current emissions, derived from food waste discharge records, and the overall average emissions (e.g., the average emissions per user). For example, if the difference between the current emissions and the overall average emissions exceeds a threshold, the environmental contribution may be adjusted upward. This adjustment method can be understood as embodying the design concept of short-term rewards, which prioritize providing immediate incentives.

[0108] As another example, the management server (10) can analyze food waste images using a deep learning model to detect foreign matter objects (e.g., bones, tissue paper, vinyl, metal, etc.). Then, the management server (10) can adjust the environmental contribution downward based on the type and number of detected foreign matter objects (e.g., the greater the number of foreign matters, the larger the size, and the higher the processing cost, the greater the adjustment). For a description of the deep learning model for detecting foreign matter objects, please refer to the description of FIG. 9.

[0109] As another example, if the user's current emissions (or emissions of a specific food item) have achieved a pre-assigned waste reduction mission (e.g., 10% reduction in emissions, 10% reduction in emissions of a specific food item, etc.), the management server (10) may adjust the environmental contribution upward.

[0110] As another example, the management server (10) can evaluate the user's level of emission improvement by comparing the user's current emission with the same user's past emission (e.g., previous emission, past average emission, past emission at the same location, etc.), and adjust the environmental contribution based on the evaluation result (e.g., the higher the level of emission improvement, the greater the upward adjustment of the environmental contribution).

[0111] As another example, the management server (10) can adjust the environmental contribution based on the difference between the user's current food waste discharge quality score and the past food waste discharge quality score (e.g., the previous score, the past average score, etc.). For details on the discharge quality score, please refer to the description in Fig. 5.

[0112] As another example, the management server (10) can adjust the environmental contribution based on the status information of food waste derived from the discharge records. This will be described in detail later with reference to FIG. 5.

[0113] As another example, the management server (10) can evaluate the level of food waste based on the amount of food discarded (i.e., current emissions) compared to the amount of food provided, and adjust the environmental contribution based on the evaluation results (e.g., the higher the level of food waste, the greater the downward adjustment of the environmental contribution). The method for obtaining information on the amount of food provided may be any method. For example, the management server (10) can capture images of dishes at the time of food provision through cameras installed within the restaurant and analyze the captured images to derive such information; however, the scope of the present disclosure is not limited thereto.

[0114] As another example, if the received emission records were collected at a specific location (e.g., a campsite, a camping ground, a local festival site, a public facility, etc.) or are related to a specific event period (e.g., a local festival period, a waste reduction campaign period), the management server (10) may adjust the environmental contribution upward or downward from the original level by taking into account the special circumstances. That is, if the food waste emission record satisfies a preset special condition, the management server (10) may calculate or adjust the user's environmental contribution by applying a weight to the food waste emission record.

[0115] As another example, the management server (10) generates unit emission information corresponding to the user's current emission behavior based on the received emission records, and analyzes the unit emission information and the user's waste emission history using a deep learning model to predict the likelihood that the current emission behavior corresponds to an intentional emission behavior. Then, the management server (10) can adjust the environmental contribution upward or downward based on the predicted results. The deep learning model for determining intentional emission behavior will be described below with reference to FIGS. 11 and 12 .

[0116] As another example, the management server (10) can adjust the environmental contribution based on the type of container containing the food waste. For example, the management server (10) can analyze the food waste image using a deep learning model (e.g., an object detection model) to determine whether the type of container containing the food waste (or food) is a reusable container (i.e., a container that can be reused multiple times) or a disposable container (i.e., a disposable container). If the container is determined to be reusable, the management server (10) can adjust the environmental contribution of the user (or a food seller using a reusable container, etc.) upward. If the environmental contribution of the user (or a food seller using a disposable container, etc.) is downward, the management server (10) can adjust the environmental contribution of the user (or a food seller using a disposable container, etc.) downward.

[0117] As another example, the management server (10) may adjust the user's environmental contribution based on various combinations of the examples described above.

[0118] Note that a downward adjustment to environmental contribution can be transformed into a penalty, and an upward adjustment to environmental contribution can be transformed into an additional reward.

[0119] Figure 5 is an exemplary flowchart illustrating a method for adjusting environmental contribution according to some embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as needed.

[0120] As illustrated in FIG. 5 , the present embodiments may begin with step S51, which analyzes food waste discharge records to derive waste status information. For example, the management server (10) may derive various waste status information by analyzing food waste images using a deep learning model. At this time, weight information (e.g., net weight, weight including dishes, etc.) and food order information may also be additionally utilized. For details on the structure and training method of the deep learning model used for food waste image analysis, please refer to the description in FIG. 9 .

[0121] The status information of food waste may include, but is not limited to, the total weight of the food waste, the type, number and weight (i.e., amount of discharge) of food items contained in the waste, and information on foreign substances (e.g., presence, type, number, size, etc.).

[0122] In step S52, a discharge quality score and / or discharge burden score are calculated based on waste status information. However, the specific score calculation method can be designed in various ways.

[0123] For example, the discharge quality score may be lower when the number of food items contained in the waste is higher. Conversely, the discharge burden score may be higher.

[0124] As another example, the discharge quality score may be lower when the number of foreign substances is greater, the size is larger, or the treatment cost is higher. Conversely, the discharge burden score may be higher.

[0125] As another example, the emission burden score can be calculated based on the amount of food waste (i.e., weight) and the item weight. The item weights can be set (or determined) based on, for example, carbon emissions, the pre-disposal price of the food waste, and / or the cost (or difficulty) of waste disposal. As a more specific example, the management server (10) can assign a higher weight to food waste with a relatively high carbon emissions. In this case, the environmental contribution of a user who produces a large amount of food waste with a high greenhouse gas emissions burden can be adjusted downward, thereby accurately and fairly calculating the environmental contribution for each user. As another example, the management server (10) can assign a higher weight to food waste with a relatively high pre-disposal price. In this case, a relative disadvantage for wasting expensive food waste can be imposed on the user. As another example, the management server (10) can assign a higher weight to food waste with a high disposal cost. In this case, the user's food waste disposal habits can be guided toward reducing the overall waste disposal cost. In this example, the emission quality score can be calculated inversely proportional to the emission burden score.

[0126] As another example, the emission quality score and / or emission burden score may be calculated based on various combinations of the examples described above.

[0127] In step S53, the user's environmental contribution is adjusted based on the emission quality score and / or emission burden score. For example, the management server (10) may increase the upward adjustment range of the environmental contribution as the emission quality score increases, and may increase the downward adjustment range of the environmental contribution as the emission burden score increases.

[0128] The method for calculating environmental contribution according to some embodiments of the present disclosure has been described with reference to FIGS. 4 and 5. As described above, by comprehensively considering various factors affecting the environment, a user's environmental contribution to food waste items can be accurately assessed.

[0129] Hereinafter, a method for calculating environmental contribution according to some other embodiments of the present disclosure will be described with reference to FIG. 6.

[0130] Figure 6 is an exemplary flowchart illustrating a method for calculating environmental contribution according to several other embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as needed.

[0131] As illustrated in FIG. 6, the present embodiments relate to a method for calculating environmental contribution for recyclable waste items.

[0132] As illustrated, the present embodiments may begin with step S61, which analyzes the user's recyclable waste discharge records to derive waste status information. Here, the waste status information may include, but is not limited to, the number and type of recyclable waste, whether foreign matter (e.g., contents, etc.) is present, whether packaging (e.g., labels, etc.) has been removed, and the degree of contamination.

[0133] The specific method of deriving waste status information can be designed in various ways.

[0134] For example, the management server (10) can derive waste status information by analyzing images of recyclable waste using a deep learning model. For more information on this deep learning model, please refer to the description of FIG. 10.

[0135] In step S62, a user's discharge quality score is calculated based on the waste status information. That is, the management server (10) can determine whether the waste status information conforms to predefined discharge guidelines / rules and calculate a discharge quality score based on this. For example, the discharge quality score may be low if foreign substances are included, if separation / classification is incorrect (i.e., if the separation / classification accuracy is low), if the contamination level is high, or if the packaging is not removed.

[0136] In step S63, the user's environmental contribution is determined based on the emission quality score. That is, the environmental contribution can be calculated in proportion to the emission quality score.

[0137] In step S64, the environmental contribution is adjusted according to preset rules.

[0138] For example, the management server (10) can compare the user's current emissions with the average emissions of all users (e.g., average emissions per time), or compare the accumulated emissions with the average accumulated emissions, and adjust the environmental contribution based on the comparison results.

[0139] As another example, the management server (10) can adjust the environmental contribution of recyclable waste items in a manner identical to / similar to the environmental contribution of food waste items. For more information, please refer to the descriptions of FIGS. 4 and 5.

[0140] The method for calculating environmental contribution according to several other embodiments of the present disclosure has been described with reference to FIG. 6. As described above, by comprehensively considering various factors affecting the environment, a user's environmental contribution for recyclable waste items can be accurately assessed.

[0141] Below, with reference to FIGS. 7 to 12, the structure, training method, and application examples of various deep learning models that can be utilized for integrated waste management will be described in detail.

[0142] First, a deep learning model (70) for weight prediction according to some embodiments of the present disclosure will be described with reference to FIGS. 7 and 8.

[0143] Figure 7 is an exemplary drawing for explaining a training method of a deep learning model (70).

[0144] As illustrated in Fig. 7, the deep learning model (70) is a model configured to receive an image of food waste as input and predict and output weight information (i.e., net weight information). This deep learning model (70) may also be named, in some cases, as a "weight prediction model," a "weight estimation model," a "weight measurement model," an "AI (Artificial Intelligence) model," a "neural network model," etc.

[0145] The deep learning model (70) can be trained by performing supervised learning using the net weight information of food waste as a label. Specifically, the management server (10) can collect discharge records (71) including food waste images and weight information through the food waste collection device (11), and calculate the net weight of food waste by subtracting the weight of the dishes from the corresponding weight information. For example, the management server (10) can detect a dish object in a food waste image using a trained dish detection model (72), and calculate the net weight by subtracting the weight of the object, but the scope of the present disclosure is not limited thereto. Next, the management server (10) can construct the deep learning model (70) by performing supervised learning using a training set (73) including food waste images and net weight information (i.e., labels). At this time, the training set (73) may further include food order information.

[0146] In some cases, the management server (10) may build a deep learning model (70) through a multi-task learning technique. For this, refer to the description of FIG. 9.

[0147] In some embodiments, the management server (10) may continuously update (or train) the deep learning model (70) in an online-learning manner. For example, as illustrated in FIG. 8 , whenever a waste disposal record (e.g., 81, 82) is received in real time, the management server (10) may assign a net weight label to the food waste image included in the waste disposal record and use the label to update the deep learning model (70). In this case, the performance of the deep learning model (70) may gradually improve over time. This online-learning method may also be applied to other deep learning models (e.g., 90, 100, etc.) described below.

[0148] A deep learning model (70) may be built for each site where a food waste collection device (11) is installed, or one deep learning model (70) may be shared between multiple sites (e.g., restaurants with identical or similar tableware, such as chain stores).

[0149] Deep learning models (i.e. trained models) can be used in a variety of ways.

[0150] For example, the deep learning model (70) may be utilized by the management server (10) to predict (or measure) the net weight of food waste. For example, the management server (10) may directly measure the net weight of food waste from subsequent food waste images through the deep learning model (70) without having to separately deduct the weight of the dishes. Alternatively, the management server (10) may utilize the deep learning model (70) only when the discharge record does not include weight information.

[0151] As another example, the deep learning model (70) may be installed in a food waste collection device (11) and utilized to predict (or measure) the net weight of food waste. In this example, the management server (10) may provide the food waste collection device (11) with a deep learning model (70) that is periodically or aperiodically updated (or trained). The management server (10) may also reduce the weight of the deep learning model (70) through a quantization technique and provide the result to the food waste collection device (11). In some cases, the management server (10) may provide the deep learning model (70) only to a food waste collection device (11) that does not have a weight sensor.

[0152] As another example, the deep learning model (70) may be utilized to verify weight information, etc., within the waste disposal record. Specifically, the management server (10) may perform verification by deducting the weight of the dishes from the corresponding weight information to calculate the net weight of the food waste and comparing it with the weight predicted by the deep learning model (70). As a result of the comparison, if the difference between the two values ​​exceeds a reference value, the management server (10) may ignore the corresponding waste disposal record or use the predicted value of the deep learning model (70) to determine the user's current waste disposal amount for the food waste item.

[0153] So far, a deep learning model (70) for weight prediction according to some embodiments of the present disclosure has been described with reference to FIGS. 7 and 8. Hereinafter, the structure, training method, and application examples of a deep learning model (90) for food waste image analysis will be described with reference to FIG. 9.

[0154] Figure 9 is an exemplary diagram showing the structure of a deep learning model (90).

[0155] As illustrated in FIG. 9, the deep learning model (90) is a model that can be utilized for food waste image analysis and can be trained using a multi-task learning technique. For example, the deep learning model (90) can be utilized to derive waste condition information from food waste images (or discharge records), and the weight prediction model (70) described above can also be understood to be included within the scope of the deep learning model (90) (i.e., the weight prediction task can be considered a subtask of food waste image analysis).

[0156] This deep learning model (90) may be referred to by various names depending on the type of target task. For example, if the target task is weight prediction, the deep learning model (90) may be referred to as a "weight prediction model," and if the target task is "foreign substance detection," the deep learning model (90) may be referred to as a "foreign substance detection model."

[0157] A deep learning model (90) may be configured to include an image encoder (91), an order encoder (92), an aggregator (93), and multiple predictors (94, 95). FIG. 9 illustrates an example in which the number of predictors is '2' (i.e., the number of related tasks is 2), but the scope of the present disclosure is not limited thereto. The number of predictors may increase or decrease depending on the number of tasks used in multi-task learning. Hereinafter, for the clarity of the present disclosure, each of the predictors (94, 95) will be referred to as a 'first predictor' and a 'second predictor'.

[0158] The image encoder (91) is a neural network module that encodes an input food waste image. The image encoder (91) can be configured to receive a food waste image and output a latent representation. Here, the latent representation may refer to a feature extracted from the image (e.g., a feature map, etc.) or an embedding (e.g., an embedding vector) of the image.

[0159] The image encoder (91) can be implemented with various types / forms of neural networks. For example, the image encoder (91) can be implemented based on a CNN (Convolutional Neural Network), a transformer (e.g., ViT (Vision Transformer), etc., but is not limited thereto.

[0160] The order encoder (92) is a neural network module that encodes food order information. The order encoder (92) can be configured to input food order information (e.g., in one-hot vector format) and output a latent representation (e.g., embedding). This food order information acts as contextual information that enables the deep learning model (90) to more accurately interpret food waste images, thereby reducing task difficulty and significantly improving model performance.

[0161] The order encoder (92) can be implemented using various types / forms of neural networks. For example, the order encoder (92) can be implemented based on a multi-layer perceptron (MLP) (or fully connected layer), but is not limited thereto.

[0162] The order encoder (92) may be omitted in some cases, in which case the aggregator (93) may also be omitted.

[0163] The aggregator (93) is a module that integrates the outputs of the encoders (91, 92). The aggregator (93) can be configured to generate an integrated representation (or integrated latent representation) by aggregating the latent representations generated from each encoder (91, 92).

[0164] The aggregator (93) may be a module that performs non-neural network operations such as concatenation, elementwise product, addition, multiplication, and averaging, or may be a neural network module based on MLP, etc.

[0165] The first predictor (94) is a neural network module that performs predictions specialized for the first task. The first predictor (94) can be configured to receive an integrated representation as input and output a prediction result according to the first task.

[0166] The first predictor (94) may be implemented based on a neural network such as MLP, but is not limited thereto. In addition, depending on the type of task, the first predictor (94) may be implemented as a classification head or a regression head. For example, if the first task is a categorical task (e.g., a multi-class classification task) that predicts (or classifies) the types of food items constituting food waste, the first predictor (94) may be implemented as a classification head, and if the first task is a numerical task that predicts the number and weight (i.e., net weight and discharge amount) of food items, the first predictor (94) may be implemented as a regression head.

[0167] The second predictor (95) is a neural network module that performs predictions specialized for a second task, different from the first task. The second predictor (95) can be configured to receive an integrated representation as input and output a prediction result according to the second task. Furthermore, the second predictor (95) can be implemented in a similar manner to the first predictor (94).

[0168] In some embodiments, the second predictor (95) may be configured to additionally receive weight information (e.g., weight including dishes, net weight, etc.) of food waste (provided that the second task is not a weight prediction task). For example, the second predictor (95) may be configured to additionally consider weight information to classify the type of food item, or to predict the number of food items, amount of waste, etc. In such a case, the weight information may serve as context information for the second task, thereby improving the performance of the deep learning model (90) for the second task.

[0169] Various tasks can be used for multi-task learning of a deep learning model (90). For example, a task for predicting the type of food items contained in food waste, a task for predicting the number of food items, a task for predicting the total weight (i.e., net weight) of food waste, a task for predicting the weight of each food item (i.e., net weight and amount of discharged waste), a task for detecting foreign substances contained in food waste, a task for predicting the presence of leftovers (i.e., waste), a task for determining abnormal discharge behavior, a task for determining intentional discharge behavior, a task for predicting a discharge burden score, etc. can be used for multi-task learning of a deep learning model (90). However, the scope of the present disclosure is not limited thereto.

[0170] For reference, examples of abnormal discharge behavior include an abnormally large amount of leftover food, excessive inclusion of foreign substances, or discharge of food items that do not exist in the food order information.

[0171] The management server (10) can build a deep learning model for each task by changing the target task, or can build a deep learning model in a form that can perform multiple tasks simultaneously through multiple predictors (e.g., 94, 95).

[0172] For example, the management server (10) can build an individual deep learning model (e.g., 70) for weight prediction by performing multi-task learning using a weight prediction task (i.e., a target task) together with other tasks.

[0173] As another example, the management server (10) can build an individual deep learning model for foreign substance detection by performing multi-task learning using the foreign substance detection task (i.e., target task) together with other tasks.

[0174] As another example, the management server (10) can build an individual deep learning model for predicting food item types by performing multi-task learning using a task for predicting food item types (i.e., a target task) together with other tasks.

[0175] As another example, the management server (10) can build an individual deep learning model for predicting the number of food items by performing multi-task learning using a task for predicting the number of food items (i.e., a target task) together with other tasks.

[0176] In some cases, the training set for the foreign substance detection task may be automatically generated. Specifically, the management server (10) may use a tableware detection model (e.g., 72) to identify a region of a tableware object within a food waste image and segment the remaining region excluding the region into multiple patches. Then, the management server (10) may perform color analysis and shape analysis on each patch to detect a foreign substance object (e.g., tissue paper has a white, irregular shape, and bones have a grayish-white, elongated shape, so these visual features can be utilized to detect a foreign substance object). Then, the management server (10) may use these detection results to automatically generate an object label (e.g., bounding box information, etc.) for the corresponding food waste image.

[0177] The above-described deep learning model (90, i.e., trained model) can be utilized in various ways (assuming that the deep learning model (90) is built to perform multiple tasks simultaneously).

[0178] For example, the management server (10) can derive various waste status information by analyzing food waste images through a deep learning model (90). Specifically, let us assume that the first predictor (94) is a module specialized in a weight prediction task, the second predictor (95) is a module specialized in a foreign substance detection task, the third predictor (not shown) is a module specialized in predicting the type of food item, and the fourth predictor (not shown) is a module specialized in predicting the weight (i.e., net weight and discharge amount) of each food item. In this case, the management server (10) inputs food waste images and food order information, etc. into the deep learning model (90), and then predicts the net weight of food waste (i.e., total amount of discharge) through the first predictor (94), detects foreign substances through the second predictor (95) (e.g., the second predictor (95) can be configured to output bounding box information and foreign substance type information), predicts the type of food item included in the food waste through the third predictor, and predicts the weight of each food item through the fourth predictor. In addition, the management server (10) can directly predict the discharge quality score (or discharge burden score) through the fifth predictor (not shown), and can determine abnormal discharge behavior or intentional discharge behavior through the sixth predictor (not shown).

[0179] So far, a deep learning model (90) for analyzing food waste images according to some embodiments of the present disclosure has been described with reference to FIG. 9. Hereinafter, a deep learning model (100) for analyzing recyclable waste images according to some embodiments of the present disclosure will be described with reference to FIG. 10.

[0180] Figure 10 is an exemplary diagram showing the structure of a deep learning model (100).

[0181] As illustrated in FIG. 10, the deep learning model (100) can be utilized for image analysis of recyclable waste, and the model (100) can also be trained using a multi-task learning technique. For example, the deep learning model (100) can be utilized to derive waste status information from images of recyclable waste (or discharge records).

[0182] This deep learning model (100) can be named in various ways depending on the type of target task. For example, if the target task is contamination prediction, the deep learning model (100) can be named as a "contamination prediction model," etc., and if the target task is "determining whether packaging has been removed," the deep learning model (100) can be named as a "packaging removal determination model," etc.

[0183] A deep learning model (100) may be configured to include an image encoder (101) and multiple predictors (102, 103). FIG. 10 illustrates an example where the number of predictors is '2' (i.e., the number of related tasks is 2), but the scope of the present disclosure is not limited thereto.

[0184] The image encoder (101) is a neural network module that encodes an input image of recyclable waste. The image encoder (101) can be configured to receive an image of recyclable waste and output a latent representation. For further information on the image encoder (101), please refer to the description of the image encoder (91) described above.

[0185] The first predictor (102) is a neural network module that performs predictions specialized for the first task. The first predictor (102) can be configured to receive a latent representation (e.g., feature vector, feature map, embedding vector, etc.) output from the image encoder (101) and output a prediction result according to the first task.

[0186] The second predictor (103) is a neural network module that performs predictions specialized for a second task, different from the first task. The second predictor (103) may be configured to receive an integrated representation as input and output a prediction result according to the second task. Furthermore, the second predictor (103) may be implemented in a manner similar to the first predictor (102).

[0187] In some embodiments, the second predictor (103) may be configured to additionally receive weight information of recyclable waste (provided that the second task is not a weight prediction task). For example, the second predictor (103) may be configured to additionally consider weight information to predict the type, quantity, etc. of recyclable waste. In such a case, the weight information may serve as context information for the second task, thereby improving the performance of the deep learning model (100) for the second task.

[0188] For the predictors (102, 103), please refer to the description of the predictors (94, 95) described above.

[0189] Various tasks can be used for multi-task learning of a deep learning model (90). For example, a task for predicting the type of recyclable waste, a task for predicting the number of recyclable waste, a task for predicting the total weight of recyclable waste, a task for predicting the weight (i.e., amount of discharge) of individual recyclable waste, a task for detecting foreign substances (e.g., contents, etc.) contained in recyclable waste, a task for determining whether packaging has been removed, a task for detecting abnormal discharge (or discharge behavior), a task for evaluating separation discharge accuracy, a task for predicting discharge quality scores (e.g., separation discharge accuracy, etc.), a task for determining abnormal discharge behavior, a task for determining intentional discharge behavior, etc. can be used for multi-task learning of a deep learning model (100). However, the scope of the present disclosure is not limited thereto.

[0190] For reference, abnormal discharge behaviors include cases where the separation discharge accuracy is abnormally low or where there is excessive inclusion of foreign substances.

[0191] The management server (10) can build a deep learning model for each task by changing the target task, or can build a deep learning model in a form that can perform multiple tasks simultaneously through multiple predictors (e.g., 94, 95). For further details, please refer to the description of Fig. 9.

[0192] The above-described deep learning model (100, i.e., trained model) can be utilized in various ways (assuming that the deep learning model (100) is built to perform multiple tasks simultaneously).

[0193] For example, the management server (10) can derive various waste status information by analyzing the image of recyclable waste through the deep learning model (100). Specifically, let's assume that the first predictor (102) is a module specialized in the task of weight prediction, the second predictor (103) is a module specialized in the task of foreign matter detection, the third predictor (not shown) is a module specialized in determining whether packaging has been removed, and the fourth predictor (not shown) is a module specialized in evaluating the accuracy of separation and discharge. In this case, the management server (10) inputs the image of recyclable waste into the deep learning model (100), and then predicts the weight (i.e., total amount of discharge) of recyclable waste through the first predictor (102), detects foreign matters through the second predictor (103), determines whether packaging of recyclable waste has been removed through the third predictor, and evaluates the accuracy of separation and discharge through the fourth predictor. In addition, the management server (10) can directly predict the emission quality score through the fifth predictor (not shown) and can also determine abnormal emission behavior or intentional emission behavior through the sixth predictor (not shown).

[0194] So far, a deep learning model (100) for analyzing images of recyclable waste according to some embodiments of the present disclosure has been described with reference to FIG. 10. Below, a deep learning model (110) for determining intentional discharge behavior according to some embodiments of the present disclosure will be described with reference to FIG. 11.

[0195] Figure 11 is an exemplary diagram showing the structure of a deep learning model (110).

[0196] As illustrated in FIG. 11, the deep learning model (110) is a model that can be utilized to determine intentional discharge behavior (i.e., intentional abnormal discharge behavior) in relation to food waste and / or recyclable waste, and can be configured to analyze waste discharge information in the form of a sequence to output the likelihood that the user's current discharge behavior corresponds to an intentional discharge behavior. This deep learning model (110) may also be named as a 'intentional discharge determination model', a 'intentional probability prediction model', a 'intentional probability estimation model', a 'intentional discharge behavior determination model', etc., depending on the case.

[0197] A deep learning model (110) can be configured to include a sequence encoder (111) and a predictor (112).

[0198] The sequence encoder (111) is a neural network module that encodes the input user's current discharge information and waste discharge history. The sequence encoder (111) may be implemented using a neural network specialized for sequence data processing, such as a recurrent neural network (RNN) or a transformer. However, the scope of the present disclosure is not limited thereto.

[0199] Current discharge information refers to unit discharge information corresponding to the user's current discharge behavior, and waste discharge history can be composed of a sequence of unit discharge information corresponding to past discharge behaviors. As described above, unit discharge information can include, but is not limited to, discharge date and time, discharge location, discharge amount, waste image, waste condition information, discharge quality score, discharge burden score, and abnormal discharge type (e.g., excessive discharge, containing foreign substances, mixed discharge, unpackaged, etc.).

[0200] The sequence encoder (111) can accurately capture discharge patterns with a high probability of intentional discharge by analyzing the sequence of such unit discharge information. For example, the sequence encoder (111) can effectively capture patterns such as repetition of the same abnormal discharge type, excessive discharge within a short period of time, discharge with an excessive proportion of foreign substances, discharge mixed with multiple abnormal types, and discharge of food items not included in the food order information.

[0201] The predictor (112) is a neural network module that predicts (or estimates) the likelihood of the current emission behavior being intentional based on the output of the sequence encoder (111). The predictor (112) may be implemented as a regression head that outputs a numerical value indicating the likelihood of intention, or as a classification head that outputs a confidence score for the level of intention. For further details on the predictor (112), please refer to the description of the predictor (e.g., 94) described above.

[0202] In some embodiments, the sequence encoder (111) and / or predictor (112) may be configured to receive additional user notification history. The notification history may include, but is not limited to, information such as the number of notifications for abnormal discharge behavior and whether notifications were received. This notification history may serve as a hint for predicting the likelihood of intentional behavior, thereby improving the performance of the deep learning model (110).

[0203] The above-described deep learning model (110) can be trained using a supervised learning method, and the training set can be configured to include a sequence of unit emission information (i.e., current emission information and waste emission history) and intentional labels.

[0204] In some cases, the management server (10) may automatically generate an intentional label. For example, the management server (10) may automatically assign an intentional label to a sequence of unit discharge information that exhibits a pattern such as repetition of the same abnormal discharge type, excessive discharge within a short period of time, discharge with an excessive proportion of foreign substances, discharge mixed with multiple abnormal types, or discharge of food items not included in the food order information.

[0205] The deep learning model described above (110, i.e., the trained model) can be utilized in various ways.

[0206] For example, the management server (10) can predict the possibility that a user's emission behavior is an intentional emission behavior through a deep learning model (110), and if the predicted intentional possibility is higher than a threshold (i.e., if the user is determined to be an intentional emission behavior), the user's environmental contribution can be adjusted downward (e.g., adjusted downward more than the original) or a penalty can be imposed. At this time, the management server (10) can also determine the adjustment range based on the difference between the predicted intentional possibility and the threshold. In addition, if the user is determined to be an intentional emission behavior, the management server (10) can also transmit a notification message including related information (e.g., an intentional emission behavior warning, penalty information, emission instructions, etc.) to the user terminal (12).

[0207] In some embodiments, as illustrated in FIG. 12, the management server (10) may sequentially perform the abnormal discharge behavior determination step and the intentional discharge behavior determination step. Specifically, the management server (10) may use an autoencoder (120) trained using normal waste images to determine (or detect) the user's abnormal discharge behavior. For example, the management server (10) may input a waste image of current discharge information (or discharge record) into the autoencoder (120) to generate a reconstruction image, and may calculate a reconstruction loss representing the difference between the input waste image and the reconstruction image. If this reconstruction loss is greater than a reference value, the management server (10) may determine the user's current discharge behavior as an abnormal discharge behavior. Next, the management server (10) obtains the waste discharge history corresponding to the past discharge behaviors of the user, and analyzes the current discharge information and the waste discharge history through a deep learning model (110) to determine whether the current discharge behavior of the user corresponds to an intentional discharge behavior.

[0208] For reference, if the user's current discharge behavior is determined to be an abnormal discharge behavior through the autoencoder (120), the management server (10) may transmit a notification message including related information (e.g., abnormal discharge warning, penalty information, discharge instructions, etc.) to the user terminal (12).

[0209] So far, a deep learning model (120) for determining intentional discharge behavior according to some embodiments of the present disclosure has been described with reference to FIGS. 11 and 12. Hereinafter, an exemplary computing device (130) capable of implementing a management server (10), etc., will be described with reference to FIG. 13.

[0210] Figure 13 is an exemplary hardware configuration diagram showing a computing device (130).

[0211] As illustrated in FIG. 13, a computing device (130) may include one or more processors (131), a bus (133), a communication interface (134), a memory (132) for loading a computer program (136) executed by the processor (131), and a storage (135) for storing the computer program (136). However, only components related to the embodiment of the present disclosure are illustrated in FIG. 13. Therefore, a person skilled in the art to which the present disclosure pertains will appreciate that, in addition to the components (131 to 135) illustrated in FIG. 13, other general components (e.g., input devices, output devices, etc.) may be further included. That is, the computing device (130) may further include various components in addition to the components (131 to 135) illustrated in FIG. 13. Additionally, in some cases, the computing device (130) may be configured in a form in which some of the components (131 to 135) illustrated in FIG. 13 are omitted. Hereinafter, each component of the computing device (130) will be described.

[0212] The processor (131) can control the overall operation of each component of the computing device (130). The processor (131) can be configured to include at least one of a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), an NPU (Neural Processing Unit), a TPU (Tensor Processing Unit), a VPU (Vision Processing Unit), an APU (Accelerated Processing Unit), or any other type of processor well known in the technical field of the present disclosure. In addition, the processor (131) can perform operations for at least one application or program for executing specific operations / steps / methods. The computing device (130) can include one or more processors.

[0213] Next, the memory (132) can store various data, commands, and / or information. The memory (132) can load a computer program (136) from the storage (135) to execute specific operations / steps / methods. The memory (132) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0214] Next, the bus (133) can provide a communication function between components of the computing device (130). The bus (133) can be implemented as various types of buses such as an address bus, a data bus, and a control bus.

[0215] Next, the communication interface (134) can support wired and wireless communication of the computing device (130). To this end, the communication interface (94) can be configured to include a communication module (e.g., an RF module supporting wireless communication) well known in the technical field of the present disclosure, and can support various communication methods.

[0216] Next, the storage (135) can non-temporarily store one or more computer programs (136). The storage (135) can be configured to include non-volatile memory such as Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0217] Next, the computer program (136) may include instructions that cause the processor (131) to perform specific operations / steps / methods when loaded into the memory (132). That is, the processor (131) may perform specific operations / steps / methods by executing the loaded instructions.

[0218] For example, the computer program (136) may include instructions for performing an operation of collecting a user's waste discharge record from a food waste collection device (11-1) and / or a recyclable waste collection device (11-2) and an operation of comprehensively managing the user's waste discharge information based on the collected waste discharge record.

[0219] As another example, the computer program (136) may include instructions to cause the management server (10) to perform at least some of the operations described with reference to FIGS. 1 to 12.

[0220] In the case illustrated, a management server (10) according to some embodiments of the present disclosure may be implemented via a computing device (130).

[0221] Meanwhile, in some embodiments, the computing device (130) illustrated in FIG. 13 may refer to a virtual machine implemented based on cloud technology. For example, the computing device (130) may be a virtual machine operating on one or more physical servers included in a server farm. In this case, at least some of the processor (131), memory (132), and storage (135) illustrated in FIG. 13 may be virtual hardware, and the communication interface (134) may also be implemented as a virtualized networking element, such as a virtual switch.

[0222] So far, with reference to FIG. 13, an exemplary computing device (130) capable of implementing a management server (10) according to some embodiments of the present disclosure has been described.

[0223] Various embodiments of the present disclosure and effects according to the embodiments have been described with reference to FIGS. 1 through 13. The effects according to the technical concept of the present disclosure are not limited to the effects described above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0224] Furthermore, even though the above embodiments have described multiple components as being combined or operating in combination, the technical concept of the present disclosure is not necessarily limited to these embodiments. That is, within the scope of the technical concept of the present disclosure, all of the components may be selectively combined and operated one or more times.

[0225] The technical concepts of the present disclosure described so far can be implemented as computer-readable code on a computer-readable recording medium. A computer program recorded on the computer-readable recording medium can be transmitted to another computing device via a network such as the Internet, installed on the device, and used therein.

[0226] Although the operations are depicted in a specific order in the drawings, it should not be understood that the operations must be performed in the specific order depicted, or in a sequential order, or that all depicted operations must be performed to achieve the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Although various embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art to which the present disclosure pertains will understand that the technical concepts of the present disclosure can be implemented in other specific forms without changing the technical concepts or essential characteristics thereof. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the technical ideas defined by the present disclosure.

Claims

1. A food waste collection device that takes a picture of one or more dishes containing leftover food to create an image of the user's food waste, measures the weight of the one or more dishes through a weight sensor to obtain weight information, and provides a food waste discharge record including the food waste image and the weight information; and Including an integrated waste management server that comprehensively manages the user's waste disposal information based on the food waste disposal record. Integrated waste management system.

2. In paragraph 1, The above food waste collection device, Receive a specific order number randomly issued by the food ordering device from the user, Transmit the above specific order number to the integrated waste management server, The above integrated waste management server, Receive order numbers and identification information matching the order numbers from the food ordering device; Obtaining the user's identification information matching the specific order number among the above identification information, The above-mentioned acquired identification information is the information entered by the user into the food ordering device during the order information entry process. Integrated waste management system.

3. In paragraph 1, It further includes a recycling waste collection device that takes a picture of the recycling waste input by the user to create an image of the recycling waste and provides a record of the discharge of the recycling waste including the image of the recycling waste. The above integrated waste management server, Manage the waste discharge information based on the above-mentioned recyclable waste discharge records, Integrated waste management system.

4. In paragraph 3, The above integrated waste management server, Calculate the user's first discharge amount for the food waste item based on the above food waste discharge record, The first environmental contribution of the user is determined by comparing the first emission amount and the first average emission amount of multiple users for the food waste item. By analyzing the above recycling waste discharge records, the discharge quality score of the above user is calculated, Based on the above discharge quality score, the second environmental contribution of the user for the recyclable waste item is determined. The user's integrated environmental contribution is determined by synthesizing the first environmental contribution and the second environmental contribution based on a weight basis, and the weight given to the first environmental contribution is higher than that of the second environmental contribution. Calculating the reward to be paid to the user based on the contribution to the above integrated environment. Integrated waste management system.

5. In paragraph 4, The above integrated waste management server, Detecting one or more foreign matter objects in the food waste image using the first deep learning model, Based on the detection results, the first environmental contribution is adjusted downward, By analyzing the image of the above recyclable waste through the second deep learning model, the discharge quality score is calculated by determining whether foreign substances are included, the degree of contamination, whether packaging has been removed, and the accuracy of separation discharge. The integrated environmental contribution is determined by synthesizing the adjusted first environmental contribution and the second environmental contribution. Integrated waste management system.

6. In paragraph 4, The above integrated waste management server, Evaluate the level of emission improvement by comparing the current emission amount derived from the above food waste emission record with the past emission amount of the user, Evaluate the level of food waste by comparing the amount of food provided to the user with the current amount of food emitted; Adjusting the first environmental contribution based on the above emission improvement level and the above food waste level, Integrated waste management system.

7. In paragraph 4, The above integrated waste management server, By analyzing the above food waste discharge records, information on one or more food items included in the above leftovers and the discharge amount of each food item is derived, Based on the amount of discharged food and the weight of each food item above, the discharge burden score for the food waste is calculated. Adjust the first environmental contribution based on the above emission burden score, The above item weights are determined based on the carbon emissions of each food item. Integrated waste management system.

8. In paragraph 7, The above food waste discharge record further includes the user's food order information, The above food waste discharge records are analyzed using a deep learning model. The above deep learning model: An image encoder that encodes the above food waste image to generate a first latent representation; An order encoder that encodes the above food order information to generate a second latent representation; An aggregator that generates an integrated representation by aggregating the first latent representation and the second latent representation; A classification head that receives the integrated expression and the weight information and predicts the type of the one or more food items; and Including a regression head that receives the integrated expression and the weight information and predicts the amount of discharge of each food item. Integrated waste management system.

9. In paragraph 1, The above integrated waste management server, Detecting one or more tableware objects in the above food waste image, The net weight of food waste is calculated by subtracting the weight of the detected tableware object from the above weight information, Train a deep learning model using the above net weight as a label for the food waste image, Predicting the net weight of food waste included in subsequent food waste images using the above trained deep learning model. Integrated waste management system.

10. In paragraph 1, The above integrated waste management server, Obtain unit emission information corresponding to the user's current emission behavior from the above food waste emission record, Based on the above unit emission information, it is determined whether the current emission behavior is an abnormal emission behavior, If determined to be an abnormal discharge behavior: Obtain the waste discharge history corresponding to the past discharge actions of the above user, By analyzing the above unit emission information and the above waste emission history through a deep learning model, the possibility that the current emission behavior is an intentional emission behavior is estimated. If the estimated probability is greater than the standard, a penalty will be imposed on the user. The above deep learning model: A sequence encoder encoding the unit discharge information and the waste discharge history; and Including a predictor that estimates the possibility based on the encoding result, Integrated waste management system.

11. In paragraph 1, The above integrated waste management server, If the above food waste discharge record satisfies the preset conditions, the user's environmental contribution is calculated by applying a weight to the above food waste discharge record, The above preset conditions are: If the above food waste discharge record is generated from the food waste collection device installed in a specific location, and Including cases where the above food waste discharge record was created during a specific event period, Integrated waste management system.

Images