Adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithm

Abstract

A self-adaptive sorting strategy optimization method based on artificial intelligence and a machine learning algorithm comprises the steps that 1, a sorting system based on multiple agents is established in a production workshop, and the sorting system comprises a plurality of auxiliary agents and a main agent located in the production workshop; 2, a reward function is constructed for each sorted sample, and dynamic weights are distributed for all indexes in the reward functions; and step 3, based on the currently established dynamic weight reward function, the slave agent carries out sorting by adopting a probability distribution-based hybrid exploration strategy. The method has the following beneficial effects: 1, the adaptability of the system is improved: by introducing sample quality stability indexes and dynamic weight distribution, the method can automatically adjust a sorting strategy to adapt to different production environments; 2, the sorting efficiency is optimized, an exploration strategy based on probability distribution and an exploration strategy based on counting are adopted, the intelligent agent can more effectively explore and learn the optimal sorting strategy, and the sorting efficiency is improved;

Description

An Adaptive Sorting Strategy Optimization Method Based on Artificial Intelligence and Machine Learning Algorithms Technical Field

[0001] This invention relates to the field of industrial automation and intelligent manufacturing technology, and to an adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms. Background Technology

[0002] In real-world industrial and agricultural production, sorting systems are crucial for improving production efficiency and product quality. Due to the complexity of the production environment and the diversity of sample physicochemical properties, traditional sorting strategies often struggle to adapt to changing production conditions, resulting in limited sorting efficiency and accuracy. With the development of artificial intelligence and machine learning technologies, researchers have begun exploring how to apply these techniques to optimize sorting strategies to improve the system's adaptability and intelligence.

[0003] Currently, the optimization of sorting strategies mainly relies on expert experience and fixed algorithms. While these methods may be effective under specific conditions, they often struggle to achieve adaptive adjustments in complex and ever-changing production environments. For example, some systems use simple rule engines to determine sorting strategies. These rule engines are designed with limited samples and environmental conditions in mind and cannot cope with unknown or changing situations. Additionally, some systems attempt to improve adaptability by adding sensors and manually adjusting parameters, but this approach is inefficient and difficult to optimize in real time.

[0004] Existing sorting systems suffer from the following problems and shortcomings when facing different production environments and sample characteristics. First, fixed algorithms cannot dynamically adjust sorting strategies, leading to performance degradation in new environments. Second, systems relying on expert experience lack sufficient flexibility and generalization ability, making it difficult to cope with unknown situations. Furthermore, manually adjusting parameters is inefficient and prone to errors. Therefore, developing an adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms is of great significance for improving the adaptability and intelligence of sorting systems. Summary of the Invention

[0005] The technical problems to be solved by this invention are: 1) Traditional sorting strategies rely on expert experience and fixed algorithms, which cannot be dynamically adjusted to adapt to complex and ever-changing production environments; 2) Existing systems experience performance degradation in new environments, lack sufficient flexibility and generalization ability, and are difficult to cope with unknown situations; 3) Manual parameter adjustment methods are inefficient and prone to errors, making it difficult to achieve real-time optimization. Therefore, this invention provides an adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms.

[0006] To address the aforementioned problems, this invention is achieved through the following method: an adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms, comprising the following steps: Step 1: Establishing a multi-agent sorting system in the production workshop. The sorting system includes multiple slave agents located on the side of the production line and a master agent located within the production workshop. The master agent is responsible for overall task allocation and coordination, while the slave agents are responsible for sorting specific areas or sample types. The master agent periodically collects environmental status and sorting experience information from the slave agents, and performs comprehensive analysis and evaluation. The slave agents communicate with each other, and with the master agent. Step 2: Constructing a reward function for each sorted sample and assigning dynamic weights to various indicators within the reward function. Step 3: The slave agents perform sorting based on the currently established dynamic weight reward function, using a probability distribution-based hybrid exploration strategy.

[0007] The agents are distributed on the production line, with each agent responsible for a different sorting area or different types of samples. The agents share environmental status and sorting experience through a communication mechanism.

[0008] When a slave agent discovers an efficient sorting strategy, it first sends the strategy information to the master agent. The master agent verifies and optimizes the strategy before broadcasting it to other slave agents. At the same time, a trust mechanism is established between agents, assigning different trust levels to slave agents based on their historical performance and accuracy. Slave agents with higher trust levels have higher priority when their strategy information is propagated.

[0009] Step 2 specifically includes: Step 2.1: Establish a reward function, where the total reward = basic sorting reward + efficiency reward + quality reward + constraint penalty; and determine the establishment index for each reward; Step 2.2: Based on the reward function established in Step 2.1, according to the attributes of the sorted samples and the characteristics of different production environments, assign dynamic weights to each index in the reward function: i) Sample quality stability index: Give extra rewards to sorting strategies that can control sample quality fluctuations within a very small range during the sorting process; ii) Energy consumption index: Increase the weight of the energy consumption index in production environments with extremely high energy consumption requirements; iii) Efficiency index: Increase the weight of the efficiency index in environments with high sorting efficiency requirements.

[0010] iv) Sorting accuracy.

[0011] Step 3 specifically includes: Step 3.1: In the process of interaction between the agent and the environment, a Boltzmann exploration strategy is first adopted; Step 3.1.1: Calculate the value Q of all actions; Step 3.1.2: Calculate the action probability distribution P; Step 3.1.3: Sample actions according to the temperature parameter τ, and dynamically adjust the temperature parameter τ; Step 3.1.3.1: Learn from the set constant temperature parameter τ for time 0 to t1; Step 3.1.3.2: After time t1, increase the temperature parameter τ to make the agent more inclined to randomly select actions for exploration; Step 3.1.3.3: As learning progresses, after time t2, t2 is greater than t1, gradually decrease the temperature parameter τ to make the agent more inclined to select actions with high estimated value; Step 3.2: Adopt a counting-based exploration strategy, give extra rewards to actions with fewer visits, and encourage the agent to explore sorting strategies that have not been tried before.

[0012] It also includes step 3.4: establishing a learning progress for each agent and dynamically adjusting the temperature parameter τ according to the learning progress, specifically including: step 3.4.1: collecting the total sorting task and the completed sorting task for each agent to obtain the learning progress of each agent 22, learning progress = completed sorting task / total sorting task; step 3.4.2: evaluating the current environmental complexity of the agent; (1) physicochemical properties of the production environment and samples: setting up environmental sensors in the production workshop, including but not limited to temperature and humidity sensors, oxygen concentration sensors, gas sensors and wind speed sensors; also including physicochemical sensors for detecting the physicochemical properties of samples; when the fluctuations of the environmental sensors and physicochemical sensors exceed When the threshold is exceeded, the environment is judged to be complex, the temperature parameter is increased, and the agent is allowed to conduct random exploration again to a certain extent; (2) Perception of the operating status of production equipment: Install various equipment status sensors on the production equipment to obtain the operating parameters of the equipment in real time. When the equipment status sensor exceeds the set threshold, the environment is judged to be complex, and the agent chooses to reduce the sorting speed to protect the equipment; Step 3.4.3: Adjust according to the learning progress of the agent and the complexity of the environment: If the learning progress is lower than the minimum progress threshold and the environment is complex, the temperature parameter is slowly reduced and random exploration continues; If the learning progress is higher than the maximum progress threshold and the environment is relatively simple, the temperature parameter is quickly reduced and the strategy convergence is accelerated, so that the agent converges to the action with high estimated value more quickly.

[0013] Compared with existing technologies, this invention has the following beneficial effects: 1. Enhanced system adaptability: By introducing sample quality stability indicators and dynamic weight allocation, this invention can automatically adjust the sorting strategy to adapt to different production environments, enhancing the system's adaptability and intelligence. 2. Optimized sorting efficiency: By adopting probability distribution-based and counting-based exploration strategies, the agent can more effectively explore and learn the optimal sorting strategy, improving sorting efficiency. 3. Dynamic adjustment of learning progress: The temperature parameters are dynamically adjusted according to the agent's learning progress and environmental complexity, enabling the agent to adopt different learning strategies at different stages, improving learning efficiency. Attached Figure Description

[0014] Figure 1 is a schematic diagram of the sorting system in this invention; Figure 2 is a flowchart of this invention; Figure 3 is a timing diagram of multi-agent strategy cooperation; Figure 4 is a flowchart of the hybrid exploration strategy of agents; Figure 5 is a flowchart of the Boltzmann exploration strategy; Figure 6 is a flowchart of dynamic temperature adjustment based on learning progress; Figure 7 is a flowchart of real-time monitoring and response to dynamic environmental changes. Detailed Implementation

[0015] The invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0017] An adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms dynamically adjusts the sorting strategy to adapt to different production environments, thereby improving the system's adaptability and intelligence.

[0018] As shown in Figure 2, step 1: Establish a multi-agent sorting system in the production workshop. As shown in Figure 1, the sorting system includes multiple slave agents 22 located on the side of production line 1, and a master agent 21 located within the production workshop. The master agent 21 is responsible for overall task allocation and coordination, while the slave agents 22 are responsible for sorting specific areas or sample types. The master agent 21 periodically collects environmental status and sorting experience information from the slave agents 22 for comprehensive analysis and evaluation. The slave agents 22 communicate with each other, and with the master agent 21.

[0019] Better still, as shown in Figure 3, the slave agents 22 are distributed on the production line 1, with each slave agent 22 responsible for different sorting areas or different types of samples. The slave agents 22 share environmental states and sorting experience through a communication mechanism. For example, when an agent finds an efficient sorting strategy in a certain production environment, it transmits the strategy information to other slave agents 22, thereby improving the overall sorting efficiency.

[0020] Better still, when a slave agent 22 discovers an efficient sorting strategy, it first sends the strategy information to the master agent 21. The master agent 21 then verifies and optimizes the strategy before broadcasting it to other slave agents 22. Simultaneously, a trust mechanism is established between the agents, assigning different trust levels to each slave agent 22 based on its historical performance and accuracy. Slave agents 22 with higher trust levels have higher priority when their strategy information is propagated.

[0021] Step 2: Construct a reward function for each sorted sample and assign dynamic weights to each indicator in the reward function; Step 2.1: Establish the reward function, total reward = basic sorting reward + efficiency reward + quality reward + constraint penalty; and determine the establishment index for each reward; Step 2.2: Based on the reward function established in Step 2.1, assign dynamic weights to each indicator in the reward function according to the attributes of the sorted sample and the characteristics of different production environments: i) Sample quality stability index: Give extra rewards to sorting strategies that can control sample quality fluctuations within a very small range during the sorting process; ii) Energy consumption index: Increase the weight of the energy consumption index in production environments with extremely high energy consumption requirements; iii) Efficiency index: Increase the weight of the efficiency index in environments with high sorting efficiency requirements.

[0022] iv) Sorting accuracy: By comprehensively considering sorting accuracy, efficiency and energy consumption, this method enables the reward function to more accurately guide the agent to learn the optimal sorting strategy suitable for different production environments.

[0023] For example, based on sorting accuracy, efficiency, and energy consumption, a sample quality stability index is introduced. The weights for sorting accuracy, efficiency, energy consumption, and sample quality stability are set at 0.5, efficiency at 0.3, energy consumption at 0.2, and sample quality stability at 0.1. In production environments with extremely high energy consumption requirements, the weight for energy consumption is increased to 0.3, the weight for sorting accuracy is decreased to 0.4, the weight for efficiency is decreased to 0.2, and the weight for sample quality stability remains unchanged.

[0024] Step 3: Based on the currently established dynamic weight reward function, agent 22 uses a hybrid exploration strategy based on probability distribution for sorting, as shown in Figure 4. The specific steps are as follows: Step 3.1: During the interaction between agent 22 and the environment, a Boltzmann exploration strategy is first adopted, as shown in Figure 5; Step 3.1.1: Calculate the value Q of all actions; Step 3.1.2: Calculate the action probability distribution P; Step 3.1.3: Sample actions according to the temperature parameter τ and dynamically adjust the temperature parameter τ; Step 3.1.3.1: Learn from the set constant temperature parameter τ for time 0 to t. 1; Step 3.1.3.2: After time t1, increase the temperature parameter τ to make the agent more inclined to randomly select actions for exploration, for example, setting the initial temperature parameter to 10; Step 3.1.3.3: As learning progresses, after time t2 (t2 is greater than t1), gradually decrease the temperature parameter τ to make the agent more inclined to select actions with high estimated value; for example, the temperature parameter is reduced by 0.1 for every 10 sorting tasks completed; Step 3.2: Adopt a counting-based exploration strategy, giving extra rewards for actions with fewer visits to encourage the agent to explore untried sorting strategies. For example, give an extra reward of 0.05 for actions with fewer visits to encourage the agent to explore untried sorting strategies.

[0025] Better still, it also includes step 3.4: establish a learning progress for each agent and dynamically adjust the temperature parameter τ according to the learning progress, as shown in Figure 6, specifically including: step 3.4.1: collect the total sorting task and the completed sorting task for each agent 22, so as to obtain the learning progress of each agent 22, learning progress = completed sorting task / total sorting task; step 3.4.2: evaluate the current environmental complexity of the agent 22; (1) production environment and physical and chemical properties of samples: during the production process, the physical and chemical properties of samples gradually change over time, and the complexity of the production environment will also change suddenly. Therefore, environmental sensors are set up in the production workshop. Environmental sensors include, but are not limited to, temperature and humidity sensors, oxygen concentration sensors, gas sensors, wind speed sensors, etc.; also include physical and chemical sensors for detecting the physical and chemical properties of samples, such as: electrochemical sensors, pH sensors, biosensors, density sensors, etc. When the fluctuations of environmental sensors and physical and chemical sensors exceed the threshold, the environment is judged to be complex, the temperature parameter is increased, and the agent is allowed to conduct a certain degree of random exploration again. For example, when a fluctuation in a key physical property of a sample is detected to exceed the threshold of ±0.1, the temperature parameter is appropriately increased by 0.1, allowing the agent to conduct a certain degree of random exploration again.

[0026] (2) Perception of Production Equipment Operating Status: In addition to considering the production environment and sample characteristics as environmental status, the perception of the operating status of the production equipment is added. Various equipment status sensors, such as temperature sensors and vibration sensors, are installed on the production equipment to acquire its operating parameters in real time. These operating parameters are also included in the description of the environmental status. When the equipment status sensor readings exceed a set threshold, the environment is considered complex, and the agent chooses to reduce the sorting speed to protect the equipment. Thus, when the agent selects a sorting strategy, it must consider not only the production environment and sample characteristics but also the operating status of the equipment, avoiding sorting errors caused by equipment malfunctions. For example, when the equipment temperature is too high, the agent can choose to reduce the sorting speed to protect the equipment.

[0027] Step 3.4.3: Adjust according to the learning progress of the agent and the complexity of the environment: If the learning progress is lower than the minimum progress threshold (30%) and the environment is complex, the temperature parameter is slowly reduced and random exploration continues. For example, the temperature parameter decreases by 0.05 for every 10 tasks completed. If the learning progress is higher than the maximum progress threshold (70%) and the environment is relatively simple, the temperature parameter is quickly reduced and the policy convergence is accelerated, so that the agent converges to the action with high estimated value more quickly. For example, the temperature parameter decreases by 0.1 for every 5 tasks completed.

[0028] Compared with existing technologies, the beneficial effects of this invention are as follows: 1. Enhanced system adaptability: By introducing sample quality stability indicators and dynamic weight allocation, this invention can automatically adjust sorting strategies to adapt to different production environments, enhancing the system's adaptability and intelligence. 2. Optimized sorting efficiency: By adopting probability distribution-based and counting-based exploration strategies, the agent can more effectively explore and learn the optimal sorting strategy, improving sorting efficiency. 3. Dynamic adjustment of learning progress: The temperature parameters are dynamically adjusted according to the agent's learning progress and environmental complexity, enabling the agent to adopt different learning strategies at different stages, improving learning efficiency. 4. Accurate reflection of learning progress: By considering the difficulty coefficient of the sorting task, the learning progress of the agent is more accurately reflected, making temperature parameter adjustments more reasonable. 5. Real-time response to environmental changes: Sensors are set up to monitor environmental parameters in real time. When significant changes occur in the environment, the agent can adjust its strategy in a timely manner to maintain sorting effectiveness. 6. Comprehensive consideration of equipment status: The perception of the operating status of production equipment is increased, allowing the agent to consider the equipment status when selecting a sorting strategy, avoiding sorting errors caused by equipment malfunctions. 7. Agent Collaboration and Sharing: Establish multiple agents and enable information sharing among them to improve overall sorting efficiency. 8. Hierarchical Collaborative Architecture: Design a hierarchical collaborative architecture among agents to optimize task allocation and coordination, improving the efficiency of policy verification and optimization. 9. Trust Mechanism Establishment: Assign trust levels based on agents' historical performance and accuracy to improve the accuracy and reliability of policy information transmission.

[0029] Due to its advanced nature, this invention has wide applications in industrial automation, agricultural production, and intelligent logistics. With the development of Industry 4.0 and intelligent manufacturing, the requirements for the intelligence and automation of production processes are increasing. This invention can effectively improve the adaptability and intelligence of sorting systems, meeting the needs of industrial automation. Simultaneously, the modernization of agricultural production also places higher demands on precision agriculture technology. The application of this invention can improve the sorting efficiency and quality of agricultural products, promoting the development of agricultural modernization. Furthermore, intelligent logistics, as a crucial component of the modern supply chain, also places high demands on the efficiency and accuracy of sorting systems. The application of this invention can improve the efficiency and accuracy of logistics sorting, promoting the development of intelligent logistics. Therefore, this invention has broad market demand and promising application prospects.

[0030] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several changes and improvements without departing from the overall concept of the present invention, and these should also be considered within the scope of protection of the present invention.

Claims

1. An adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms, characterized in that: The process includes the following steps: Step 1: Establish a multi-agent sorting system in the production workshop. The sorting system includes multiple slave agents (22) located on the side of the production line (1) and a master agent (21) located in the production workshop. The master agent (21) is responsible for overall task allocation and coordination, while the slave agents (22) are responsible for sorting specific sorting areas or sample types. The master agent (21) regularly collects environmental status and sorting experience information from the slave agents (22) and performs comprehensive analysis and evaluation. The slave agents (22) communicate with each other and with the master agent (21). Step 2: Construct a reward function for each sorted sample and assign dynamic weights to each indicator in the reward function. Step 3: The slave agents (22) sort the samples using a hybrid exploration strategy based on probability distribution based on the currently established dynamic weight reward function.

2. The adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms according to claim 1, characterized in that: The agents (22) are distributed on the production line (1). Each agent (22) is responsible for different sorting areas or different types of samples, and the agents (22) share environmental status and sorting experience through a communication mechanism.

3. The adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms according to claim 1, characterized in that: When a slave agent (22) discovers an efficient sorting strategy, it first sends the strategy information to the master agent (21). The master agent (21) verifies and optimizes the strategy, and then broadcasts it to other slave agents (22). At the same time, a trust mechanism is established between agents. Different trust levels are assigned to slave agents (22) based on their historical performance and accuracy. The strategy information of slave agents (22) with higher trust levels has higher priority when it is propagated.

4. The adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms according to claim 1, characterized in that: Step 2 specifically includes: Step 2.1: Establish a reward function, where the total reward = basic sorting reward + efficiency reward + quality reward + constraint penalty; and determine the establishment index for each reward; Step 2.2: Based on the reward function established in Step 2.1, according to the attributes of the sorted samples and the characteristics of different production environments, assign dynamic weights to each index in the reward function: i) Sample quality stability index: Give extra rewards to sorting strategies that can control sample quality fluctuations within a very small range during the sorting process; ii) Energy consumption index: Increase the weight of the energy consumption index in production environments with extremely high energy consumption requirements; iii) Efficiency index: Increase the weight of the efficiency index in environments with high sorting efficiency requirements; iv) Sorting accuracy.

5. The adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms according to claim 1, characterized in that: Step 3 specifically includes: Step 3.1: In the process of interaction between the agent and the environment, a Boltzmann exploration strategy is first adopted; Step 3.1.1: Calculate the value Q of all actions; Step 3.1.2: Calculate the action probability distribution P; Step 3.1.3: Sample actions according to the temperature parameter τ, and dynamically adjust the temperature parameter τ; Step 3.1.3.1: Learn from the set constant temperature parameter τ for time 0 to t1; Step 3.1.3.2: After time t1, increase the temperature parameter τ to make the agent more inclined to randomly select actions for exploration; Step 3.1.3.3: As learning progresses, after time t2, t2 is greater than t1, gradually decrease the temperature parameter τ to make the agent more inclined to select actions with high estimated value; Step 3.2: Adopt a counting-based exploration strategy, give extra rewards to actions with fewer visits, and encourage the agent to explore sorting strategies that have not been tried before.

6. The adaptive sorting strategy optimization method based on artificial intelligence and machine learning algorithms according to claim 5, characterized in that: It also includes step 3.4: establishing a learning progress for each agent and dynamically adjusting the temperature parameter τ according to the learning progress, specifically including: step 3.4.1: collecting the total sorting task and the completed sorting task for each agent to obtain the learning progress of each agent 22, learning progress = completed sorting task / total sorting task; step 3.4.2: evaluating the current environmental complexity of the agent; (1) physicochemical properties of the production environment and samples: setting up environmental sensors in the production workshop, including but not limited to temperature and humidity sensors, oxygen concentration sensors, gas sensors and wind speed sensors; also including physicochemical sensors for detecting the physicochemical properties of samples; when the fluctuations of the environmental sensors and physicochemical sensors exceed When the threshold is exceeded, the environment is judged to be complex, the temperature parameter is increased, and the agent is allowed to conduct random exploration again to a certain extent; (2) Perception of the operating status of production equipment: Install various equipment status sensors on the production equipment to obtain the operating parameters of the equipment in real time. When the equipment status sensor exceeds the set threshold, the environment is judged to be complex, and the agent chooses to reduce the sorting speed to protect the equipment; Step 3.4.3: Adjust according to the learning progress of the agent and the complexity of the environment: If the learning progress is lower than the minimum progress threshold and the environment is complex, the temperature parameter is slowly reduced and random exploration continues; If the learning progress is higher than the maximum progress threshold and the environment is relatively simple, the temperature parameter is quickly reduced and the strategy convergence is accelerated, so that the agent converges to the action with high estimated value more quickly.

Images