Skid type vehicle body storage area scheduling method based on reinforcement learning

Abstract

The invention provides a skid type vehicle body storage area scheduling method based on reinforcement learning. The method comprises the steps of obtaining a real-time production plan of a vehicle body production line; performing tensorization processing on the plurality of pieces of production plan data to obtain a plurality of pieces of corresponding tensorized production plan data; according to the multiple pieces of tensorized production plan data, respectively determining output sequence attributes of storage channels, output sequence attributes of input channels and output sequence attributes of output channels in the skid type vehicle body storage area; the instantaneous state attribute of the skid type vehicle body storage area is constructed; according to the instantaneous state attributes, adopting a preset reinforcement learning model to generate an optimal scheduling sequence of the skid type vehicle body storage area, and according to the optimal scheduling sequence, the output sequence attributes of the multiple target storage channels, the output sequence attributes of the input channels and the output sequence attributes of the output channels, obtaining the optimal scheduling sequence of the skid type vehicle body storage area. And an action instruction of the corresponding target transition machine equipment is generated, and efficient cooperation of vehicle body storage and vehicle body production is achieved.

Description

Technical Field

[0001] This invention relates to the field of vehicle body control technology, and more specifically, to a skid-type vehicle body storage area scheduling method based on reinforcement learning. Background Technology

[0002] In the domestic commercial vehicle sector, existing production models need to address seasonal demand fluctuations, over a hundred welding types (corresponding to thousands of vehicle models), a fixed process sequence of "welding first, painting second, and final assembly" with welding assembly, painting color matching, and final assembly, as well as painting or final assembly delays caused by inspection and production scheduling in the welding, painting, and final assembly stages. Currently, the industry mostly uses the body-in-white storage area to make basic adjustments to the differences in the welding and painting production sequence. For example, it uses manual scheduling of body positions within the body-in-white storage area based on experience, or relies on a simple "first-in, first-out" rule to arrange the entry of bodies into the painting process. At the same time, it manually counts the number of empty skids and return paths, and manually coordinates the data interaction between the transfer equipment and the scheduling system to adapt to production needs.

[0003] However, existing technologies have significant shortcomings. Manual scheduling and simple rules cannot respond in real time to fluctuations in production during peak and off-peak seasons and the demand for multi-variety production. Mismatches often occur between the order of car bodies in the body-in-white storage area and the painting plan, resulting in waiting or out-of-order painting processes, making it difficult to efficiently bridge the differences between welding and painting production. On the other hand, there is a lack of intelligent integration of the physical structure, equipment and system coordination of the body-in-white storage area. It is impossible to accurately calculate the distance from the car body in the warehouse to the exit, dynamically optimize the car body storage position, and make decisions based on actual production data. Ultimately, the buffering effect of the body-in-white storage area fails, affecting overall production efficiency. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the prior art by providing a skid-type vehicle body storage area scheduling method based on reinforcement learning. This method enables the real-time acquisition of production plans and the processing of structured data to accurately map the relationship between each channel of the skid-type vehicle body storage area and the output order. It also dynamically generates the optimal scheduling strategy by combining a preset reinforcement learning model, and finally transforms it into action instructions for the target transfer machine equipment.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for scheduling storage areas in a skid-type vehicle based on reinforcement learning, the method comprising: Obtain the real-time production plan of the vehicle body production line, wherein the real-time production plan includes: multiple production plan data corresponding to multiple vehicles to be processed; Tensorization processing is performed on the multiple production plan data to obtain corresponding multiple tensorized production plan data; each tensorized production plan data includes: the body information and sequence number of the multiple vehicles to be processed; Based on the multiple production plan data, the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type body storage area are determined respectively. Based on the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel, the instantaneous state attributes of the skid-type vehicle body storage area are constructed. Based on the instantaneous state attributes, a preset reinforcement learning model is used to generate the optimal scheduling order of the skid-type vehicle body storage area. The optimal scheduling order includes the scheduling order of multiple target storage channels. Based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels, corresponding action instructions for the target transfer device are generated.

[0006] In an optional implementation, each production plan data includes: original part number, original color design, original status code, original material version, original date, original batch sequence, and sequence information; the body information of the multiple bodies to be processed includes: quantified part number, quantified color design, quantified status code, quantified material version, quantified original date, quantified original batch, and quantified original single-unit sequence. The tensor quantization process of the multiple production plan data to obtain corresponding tensor quantized production plan data includes: Perform hash modulo operations on the original part number and the original color design respectively to obtain the quantized part number and the quantized color design; The original state code is truncated to obtain the tensor state code; The original material version and the original date are respectively converted into numerical values ​​to obtain the quantified material version and the quantified original date. The original batch sequence is decomposed to obtain the tensor-quantized original batch and the tensor-quantized original single-unit sequence; The sequence information is converted into a sequence number to obtain the sequence number.

[0007] In an optional implementation, the output order attribute of each storage channel includes: a first exit distance range of each storage channel when it is a transmitting channel, and a second exit distance range of each storage channel when it is a receiving channel; the output order attribute of the input channel includes: the distance range of the input channel when it is a transmitting channel; and the output order attribute of the output channel includes: the exit distance range of the output channel when it is a receiving channel. The step of determining the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type vehicle body storage area based on the multiple tensile production plan data includes: Based on the multiple production plan data, the first exit distance range of each storage channel when used as a sending channel is obtained. The first exit distance range includes the sequence number of all vehicles to be processed before each vehicle to be processed on each storage channel. Based on the multiple production plan data, the second exit distance range of each storage channel when used as a receiving channel is obtained. The second exit distance range includes the sequence number of all vehicles to be processed before each vehicle to be processed on each storage channel. Based on the multiple production plan data, the distance range of the input channel when it is used as the sending channel is obtained. The distance range of the input channel includes: the sequence number of all vehicles to be processed before each vehicle to be processed on the input channel. Based on the multiple production plan data, the exit distance range of the output channel when it is used as the receiving channel is obtained. The exit distance range of the output channel includes the sequence number of all vehicles to be processed before each vehicle to be processed on the output channel.

[0008] In an optional implementation, the preset reinforcement learning model includes: Long Short Memory Network, Graph Neural Network, and Proximal Policy Optimization (PPO) algorithm; Before generating the optimal scheduling order of the skid-type vehicle body storage area using a preset reinforcement learning model based on the instantaneous state attributes, the method further includes: Based on the positional relationships between the input channel, storage channel, and output channel in the skid-type vehicle body storage area, an adjacency matrix of the skid-type vehicle body storage area is constructed. The step of generating the optimal scheduling order of the skid-type vehicle body storage area based on the instantaneous state attributes and using a preset reinforcement learning model includes: The instantaneous state attributes are tensor-compressed using the Long Short-Term Memory network to obtain compressed state attributes. Based on the compressed state attributes and the adjacency matrix, the graph neural network is used to perform the transformation to obtain the transformed state attributes; Based on the transformed state attributes, the PPO algorithm is used to generate the optimal scheduling order for the skid-type vehicle body storage area.

[0009] In an optional implementation, before generating the corresponding action command for the target transfer device based on the optimal scheduling order, the output order attributes of the plurality of target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels, the method further includes: Obtain the action list of the target transfer machine device. The action list of the target transfer machine device records multiple action information. Each action information is used to instruct the transfer machine device to transport the vehicle body to be processed in a sending channel to a receiving channel. The step of generating corresponding action instructions for the target mobile device based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels includes: Based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer device, the corresponding action instructions for the target transfer device are generated.

[0010] In an optional implementation, before obtaining the action list of the target transfer device, the method further includes: Based on the delivery range of each transfer machine within the skid-type vehicle storage area, the skid-type vehicle storage area is divided to obtain the movement area information of each transfer machine. The movement area information includes: the information of the sending channel corresponding to each transfer machine and the information of the receiving channel corresponding to each transfer machine. Based on the movement area information of each transfer machine, an action list for each transfer machine is generated.

[0011] In an optional implementation, generating corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of the plurality of target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer device includes: If the first exit distance range of the sending channel corresponding to the target transfer device is not within the first exit distance range, then the first target sending channel is determined according to the output order attribute in the sending channel corresponding to the target transfer device; wherein, the target sending channel is the first sending channel with the smallest exit distance range; The first target receiving channel is determined based on the output order attribute in the receiving channel corresponding to the target transfer device; The action instructions for the target transfer device are determined based on the first target sending channel and the first target receiving channel.

[0012] In an optional implementation, the step of generating corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of the plurality of target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer device further includes: If the first exit distance range of the sending channel corresponding to the target mobile device is within the first exit distance range, then determine whether the first exit distance range of the sending channel corresponding to the target mobile device is within the second exit distance range; If it is determined that the first exit distance in the sending channel corresponding to the target mobile device is within the second exit distance preset range, then it is determined whether the receiving channel corresponding to the target mobile device includes a return channel, and whether the sending channel and receiving channel corresponding to the target mobile device are fully loaded. If the receiving channel corresponding to the target mobile device includes the return channel, and the sending and receiving channels corresponding to the target mobile device are fully loaded, then the return channel is determined as the second target receiving channel; and the second target sending channel is determined according to the output order attribute in the sending channel corresponding to the target mobile device. The action instructions for the target transfer device are determined based on the second target sending channel and the second target receiving channel.

[0013] In an optional implementation, the method further includes: Obtain the execution status of the action command; If the execution status of the action command is in the non-execution state, then the operating status of the target mobile device is determined; If the target mobile device is in an idle state, the action command is sent to the target mobile device, the execution status of the action command is updated to executed, and the working status of the target mobile device is updated to running. If the working state of the target transfer device changes to an idle state, it is determined that the target transfer device has completed its execution, and the execution status of the action command is updated to the execution completed state.

[0014] In an optional implementation, the method further includes: Obtain the preset empty skid plan information; According to the preset empty skid plan information, the empty skids to be transferred in the skid-type vehicle body storage area are sorted to obtain the transfer order of the empty skids to be transferred. The empty sleds to be transferred are scheduled according to the transfer order.

[0015] Secondly, embodiments of this application also provide a skid-type vehicle body storage area scheduling device based on reinforcement learning, comprising: The acquisition module is used to acquire the real-time production plan of the car body production line. The real-time production plan includes multiple production plan data corresponding to multiple cars to be processed. The processing module is used to perform tensor quantization processing on the multiple production plan data to obtain corresponding multiple tensor quantized production plan data; each tensor quantized production plan data includes: the body information and sequence number of the multiple vehicles to be processed; The determination module is used to determine the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type body storage area based on the multiple tensile production plan data. The construction module is used to construct the instantaneous state attributes of the skid-type vehicle body storage area based on the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel. The generation module is used to generate the optimal scheduling order of the skid-type vehicle body storage area based on the instantaneous state attributes and using a preset reinforcement learning model. The optimal scheduling order includes the scheduling order of multiple target storage channels. The generation module is used to generate corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels.

[0016] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the reinforcement learning-based skid-type vehicle body storage area scheduling method as described in any of the first aspects.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the skid-type vehicle body storage area scheduling method as described in any of the first aspects.

[0018] The beneficial effects of this application are: This application provides a reinforcement learning-based scheduling method for skid-type vehicle body storage areas, comprising: acquiring a real-time production plan of a vehicle body production line, the real-time production plan including: multiple production plan data corresponding to multiple vehicles to be processed; performing tensor quantization processing on the multiple production plan data to obtain corresponding multiple tensor quantized production plan data; each tensor quantized production plan data including: vehicle body information and sequence number of multiple vehicles to be processed; determining the output order attribute, input channel output order attribute, and output channel output order attribute of each storage channel in the skid-type vehicle body storage area according to the multiple tensor quantized production plan data; constructing the instantaneous state attribute of the skid-type vehicle body storage area according to the output order attribute of each storage channel, input channel output order attribute, and output channel output order attribute; generating the optimal scheduling order of the skid-type vehicle body storage area using a preset reinforcement learning model based on the instantaneous state attribute, the optimal scheduling order including: the scheduling order of multiple target storage channels; and generating corresponding action instructions for target transfer equipment according to the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels.

[0019] The method described in this application acquires production plans in real time and performs data structuring to accurately map the relationship between each channel and output sequence in the skid-type body storage area. Combined with a pre-set reinforcement learning model, it dynamically generates the optimal scheduling strategy, which is ultimately transformed into action instructions for the target transfer machine. This enables efficient collaboration between body storage and body production. Furthermore, by responding to production demands in real time and intelligently optimizing the storage scheduling sequence, it effectively balances the production rhythm between workshops, reduces production line waiting times or congestion in the skid-type body storage area caused by sequence mismatches, and improves the utilization rate of transfer machine equipment and production smoothness. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a skid-type vehicle body storage area provided in an embodiment of this application; Figure 2 One of the flowcharts for a vehicle body storage area scheduling method based on reinforcement learning provided in this application embodiment; Figure 3 A second flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, provided as an embodiment of this application; Figure 4 The third flowchart illustrates a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment. Figure 5 The fourth flowchart illustrates a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment. Figure 6 The fifth flowchart illustrates a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment. Figure 7 A flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, provided in an embodiment of this application, is shown in Figure 6. Figure 8 The seventh flowchart illustrates a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment. Figure 9 This is the eighth flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in an embodiment of this application. Figure 10 A flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, provided in an embodiment of this application, is shown in Figure 9. Figure 11 A flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, provided in this application embodiment; Figure 12 A schematic diagram of the functional modules of a skid-type vehicle body storage area scheduling device based on reinforcement learning provided in an embodiment of this application; Figure 13 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0023] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0025] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0027] In the body production line, which includes welding, painting, and final assembly processes, various inspections and production sequence adjustments occur during each process. To avoid delays or lags in the painting or final assembly processes, this application utilizes a skid-type body storage area to bridge the gaps in the welding, painting, and final assembly production sequences. The skid-type body storage area includes a White Body Storage Area (WBS), which is the welded body storage area, and a Painted Body Store (PBS). The welding workshop produces the white body, i.e., the welded body frame. The white body is first placed in the White Body Storage Area. Then, according to the painting plan, the white bodies are sequentially output to the painting workshop. After painting, the white bodies are obtained and placed in the Painted Body Storage Area. Finally, according to the final assembly plan, the painted bodies are sequentially output to the final assembly workshop.

[0028] The physical structures of the body-in-white storage area and the painted body storage area are similar, both including storage areas and transfer equipment. Therefore, in the subsequent embodiments of this application, the body-in-white storage area WBS is specifically used as an example to explain the skid-type body storage area scheduling method. Figure 1 This is a schematic diagram of the structure of a body-in-white storage area provided in an embodiment of this application, as shown below. Figure 1 As shown, the body-in-white storage area is divided into 3 zones and 8 transfer machines. The 3 zones are A (coarse sorting zone 1), B (coarse sorting zone 2), and C (fine sorting zone 1). Each zone has 5 normal channels, 1 fast channel, and 1 return channel, resulting in a total of 21 channels in the body-in-white storage area. The 8 transfer machines are also transfer machines, namely P01, P02, P03, P04, P05, P06, P07, and P08. Input channels, output channels, and inter-zone communication are handled by 2 transfer machines for moving body-in-white components.

[0029] For example, A. Coarse sorting zone 1 includes 5 normal channels: W1A01, W1A02, W1A03, W1A05 and W1A06, 1 fast channel: W1A07, and 1 return channel: W1A04; B. Coarse sorting zone 2 includes 5 normal channels: W1B01, W1B02, W1B03, W1B05 and W1B06, 1 fast channel: W1B07, and 1 return channel: W1B04; C. Fine sorting zone 1 includes 5 normal channels: W1C01, W1C02, W1C03, W1C05 and W1C06, 1 fast channel: W1C07, and 1 return channel: W1C04.

[0030] The selectable movement range of transfer machine P01 is between the input channel PIN and channels (W1A01, W1A02, W1A03, W1A04); the selectable movement range of transfer machine P02 is between the input channel PIN and channels (W1A04, W1A05, W1A06, W1A07); the selectable movement range of transfer machine P03 is between channels (W1A01, W1A02, W1A03, W1A04) and channels (W1B01, W1B02, W1B03, W1B04); the selectable movement range of transfer machine P04 is between channels (W1A04, W1A05, W1A06, W1A07) and channels (W1B04, W1B05, W1B06, W1B07); The selectable movement range of the transfer device P05 is between channels (W1B01, W1B02, W1B03, W1B04) and channels (W1C01, W1C02, W1C03, W1C04); the selectable movement range of the transfer device P06 is between channels (W1B04, W1B05, W1B06, W1B07) and channels (W1C04, W1C05, W1C06, W1C07); the selectable movement range of the transfer device P07 is between channels (W1C01, W1C02, W1C03, W1C04, W1C05, W1C06) and the output channel POUT; the selectable movement range of the transfer device P08 is between channels (W1C06, W1C07) and the output channel POUT.

[0031] This can be understood as follows: the normal channel is also a storage channel used to store the body-in-white, the fast channel is used to quickly transport the body-in-white, and the return channel is used to receive the returned body-in-white. For example, channel W1A04 can receive the body-in-white transported from channel W1A01 by transfer equipment P03 and send it to transfer equipment P01, which will then transport the body-in-white to channels W1A01, W1A02, or W1A03.

[0032] This application provides a reinforcement learning-based scheduling method for skid-type vehicle body storage areas, enabling orderly scheduling of skid-type vehicles and avoiding delays during production. The following detailed explanation, in conjunction with the accompanying drawings and specific examples, illustrates this reinforcement learning-based skid-type vehicle body storage area scheduling method. Figure 2 This application provides one of the flowcharts illustrating a reinforcement learning-based skid-type vehicle body storage area scheduling method; as shown below. Figure 2 As shown, the method includes: S101. Obtain the real-time production plan for the body production line.

[0033] The real-time production plan includes multiple production plan data corresponding to multiple vehicles awaiting processing.

[0034] S102. Perform tensor quantization on multiple production plan data to obtain corresponding tensor quantized production plan data.

[0035] Each piece of quantitative production plan data includes: body information and sequence number of multiple vehicles to be processed.

[0036] In this embodiment, the vehicle body production line includes a painting production line and a final assembly production line, and the corresponding real-time production plan includes a real-time painting plan and a real-time final assembly plan. The painting production line obtains painting plan data for each vehicle body to be processed, based on the part number corresponding to each vehicle body marker and the color information to be painted. Similarly, the final assembly production line obtains final assembly plan data for each vehicle body to be processed, based on the part number corresponding to each vehicle body marker and the color information to be assembled. Since the production plan data is sparse and infinite, tensor quantization is performed on each production plan data to facilitate subsequent model processing, resulting in identifiable tensorized production plan data. The sequence number indicates the order in which the vehicles to be processed need to be output; for example, if the sequence number is 2, it indicates that the vehicle body to be processed needs to be the second one output from the skid-type vehicle body storage area.

[0037] S103. Based on multiple production plan data, determine the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type body storage area.

[0038] S104. Construct the instantaneous state attributes of the skid-type vehicle body storage area based on the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel.

[0039] If the skid-type body storage area is a body-in-white storage area, then the multiple storage channels are for storing bodies to be painted, the input channel is for the bodies to be painted to enter the body-in-white storage area from the welding workshop, and the output channel is for the bodies to be painted to enter the painting workshop from the body-in-white storage area. If the skid-type body storage area is a painted body storage area, then the multiple storage channels are for storing bodies to be assembled, the input channel is for the bodies to be assembled to enter the painted body storage area from the painting workshop, and the output channel is for the bodies to be assembled to enter the final assembly workshop from the painted body storage area.

[0040] Based on multiple production plan data corresponding to multiple vehicles to be processed, the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel are determined respectively.

[0041] Then, the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel are concatenated to obtain the instantaneous state attributes of the skid-type vehicle body storage area.

[0042] S105. Based on the instantaneous state attributes, a preset reinforcement learning model is used to generate the optimal scheduling order for the skid-type vehicle body storage area.

[0043] The optimal scheduling order includes the scheduling order of multiple target storage channels.

[0044] The pre-defined reinforcement learning model processes the instantaneous state attributes to obtain the optimal scheduling order of the skid-type vehicle body storage area. Since there are transfer machines between multiple target storage channels, it can also be understood as determining the scheduling order of the transfer machines, that is, determining one transfer machine from eight transfer machines to be scheduled first, and then determining another transfer machine to be scheduled.

[0045] S106. Generate the corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels.

[0046] Specifically, since the model only determines the optimal scheduling order, and there are multiple sending and receiving channels corresponding to the transfer machine equipment, it is also necessary to determine the action instructions of the target transfer machine equipment, i.e., the target sending channel and target receiving channel corresponding to the target transfer machine equipment, based on the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels. For example, the action instruction is [W1C01:P07:POUT], which means that the target transfer machine equipment P07 goes to channel W1C01, picks up the first vehicle body to be processed, and then sends it to the output channel POUT. Therefore, the target sending channel is channel W1C01, and the target receiving channel is the output channel POUT. This achieves the scheduling of multiple vehicles to be processed in the skid-type vehicle body storage area.

[0047] In summary, this application provides a reinforcement learning-based scheduling method for skid-type vehicle body storage areas, comprising: acquiring a real-time production plan of a vehicle body production line, the real-time production plan including: multiple production plan data corresponding to multiple vehicles to be processed; performing tensor quantization processing on the multiple production plan data to obtain corresponding multiple tensor quantized production plan data; each tensor quantized production plan data including: vehicle body information and sequence number of multiple vehicles to be processed; determining the output order attribute, input channel output order attribute, and output channel output order attribute of each storage channel in the skid-type vehicle body storage area according to the multiple tensor quantized production plan data; constructing the instantaneous state attribute of the skid-type vehicle body storage area according to the output order attribute of each storage channel, input channel output order attribute, and output channel output order attribute; generating the optimal scheduling order of the skid-type vehicle body storage area using a preset reinforcement learning model based on the instantaneous state attribute, the optimal scheduling order including: the scheduling order of multiple target storage channels; and generating corresponding action instructions for target transfer equipment according to the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels.

[0048] The method in this application acquires the vehicle body production plan in real time and performs data structuring to accurately map the relationship between each channel and the output sequence in the skid-type vehicle body storage area. Combined with a pre-set reinforcement learning model, it dynamically generates the optimal scheduling strategy, which is ultimately transformed into action instructions for the target transfer machine. This enables efficient collaboration between vehicle body storage and production. Furthermore, by responding to production demands in real time and intelligently optimizing the storage scheduling sequence, it effectively balances the production rhythm between workshops, reduces production line waiting times or congestion in the skid-type vehicle body storage area caused by sequence mismatches, and improves the utilization rate of the transfer machine and the smoothness of production.

[0049] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Each production plan data entry includes: original part number, original color design, original status code, original material version, original date, original batch sequence, and sequence information; the vehicle body information for multiple vehicles to be processed includes: quantified part number, quantified color design, quantified status code, quantified material version, quantified original date, quantified original batch, and quantified original single-vehicle sequence. Figure 3 This is a second flowchart illustrating a reinforcement learning-based skid-type vehicle body storage area scheduling method provided in an embodiment of this application; as shown below. Figure 3 As shown, tensor quantization is performed on multiple production plan data to obtain corresponding tensor quantized production plan data, including: S201. Perform hash modulo operations on the original part number and the original color design to obtain the quantized part number and the quantized color design.

[0050] S202. Encode and truncate the original state code to obtain the tensor state code.

[0051] S203. Perform numerical conversion on the original material version and original date to obtain the Zhang quantified material version and Zhang quantified original date.

[0052] S204. Decompose the original batch sequence to obtain the tensor-quantized original batch and the tensor-quantized original single-unit sequence.

[0053] S205. Perform sequence number conversion on the sequence information to obtain the sequence number.

[0054] In this embodiment, each production plan data entry includes: original part number, original color design, original status code, original material version, original date, original batch sequence, and sequence information. For example, taking painting plan data as the production plan data, the painting plan data for a car body to be processed includes: original part number: 5000014-G14103A, original color design: T0303, original status code: 0403, original material version: 2025 / 9 / 29, original date: 2025 / 10 / 7, and original batch sequence: CX004. The sequence information includes the latest date and the latest batch sequence; therefore, the latest date is 2025 / 10 / 9, and the latest batch sequence is AG004.

[0055] Since the original part numbers all start with 5000014-, we extract G14103A starting from the 8th digit, convert it to a hash value, and then take the modulo of 99999999 and divide by 10000 to get the value 7567.9648, which is used as the quantized part number. Since the original color design only has 6 digits, we can directly convert T0303 to a hash value, and then take the modulo of 9999999 and divide by 10000 to get the value 5633.5998, which is used as the quantized color design.

[0056] Since there are only 5 original state codes, including 0401-0405, it is difficult to increase the original state codes and the increase is controllable. Therefore, the last bit of the original code is directly truncated to obtain the value 3 as the tensor state code.

[0057] The original material version's date format is directly converted to the number 20250929 and divided by 10000 to obtain 2025.0929, which is used as the quantified material version. The original date's date format is directly converted to the number 20251007 and divided by 10000 to obtain 2025.1007, which is used as the quantified original date.

[0058] The format of the original batch sequence is fixed as 2 letters and 3 numbers. CX004 is then decomposed into two parts: CX and 004. The letter CX part is converted into a hash value and modulo 999999, then divided by 1000 to get the value 736.316, which is used as the original batch for quantization. The number 004 part is directly deducted from the zeros and only the integer part is taken to get 4, which is used as the original single-unit sequence for quantization.

[0059] For sequential information, the latest date and the latest batch number are converted into 1 as the sequence number in ascending order. If there are N painting plan data, the sequence number is a value from 1 to N.

[0060] The Zhang quantitative part number in the vehicle information of the vehicle to be processed is: 7567.9648, Zhang quantitative color design is: 5633.5998, Zhang quantitative status code is: 3, Zhang quantitative material version is: 2025.0929, Zhang quantitative original date is: 2025.1007, Zhang quantitative original batch is: 736.316, and Zhang quantitative original single unit sequence is: 4. The sequence number is: 1.

[0061] The method provided in this application involves performing hash modulo operations on the original part number and original color design to obtain a quantified part number and a quantified color design; truncating the original status code to obtain a quantified status code; converting the original material version and original date to obtain a quantified material version and a quantified original date; decomposing the original batch sequence to obtain a quantified original batch and a quantified original single-unit sequence; and converting the sequence information into a sequence number. Different data are truncated and hashed using different methods to reduce the original data. The combined use of the processed quantified data eliminates the possibility of duplicate original data after hash modulo and reduction, thus realizing the conversion of production plan data into quantified production plan data.

[0062] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. The output order attributes of each storage channel include: a first exit distance range for each storage channel when used as a sending channel, and a second exit distance range for each storage channel when used as a receiving channel. The output order attributes of the input channels include: the distance range of the input channels when used as sending channels. The output order attributes of the output channels include: the exit distance range of the output channels when used as receiving channels. Figure 4 This is the third flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment; Figure 4 As shown, based on multiple quantitative production plan data, the output order attributes of each storage channel, the output order attributes of the input channels, and the output order attributes of the output channels in the skid-type vehicle body storage area are determined, including: S301. Based on multiple sheet-quantitative production plan data, obtain the first exit distance range of each storage channel when used as a sending channel.

[0063] The first exit distance range includes the exit distance of each vehicle body to be processed on each storage channel.

[0064] In this embodiment, each vehicle body to be processed has a corresponding sequence number, which is then referred to... Figure 1 Taking channel W1A01 as an example, if channel W1A01 stores a vehicle body with a sequence number of 2, then when channel W1A01 is determined as the sending channel, how many vehicles body with sequence number 2 need to be output before the vehicle body with sequence number 2 is output, and what are their corresponding sequence numbers?

[0065] Among them, channel W1A01 serves as the sending channel. There are 14 storage channels between channel W1A01 and the exit channel, namely channels W1B01-WB07 and channels W1C01-WC07. If channel W1C01 stores multiple vehicles to be processed with sequence numbers 12, 10, 8, and 6; channel W1C02 stores one vehicle to be processed with sequence number 5; channel W1C03 stores one vehicle to be processed with sequence number 3; channel W1C05 stores multiple vehicles to be processed with sequence numbers 11, 9, 7, and 4; and channel W1C06 stores one vehicle to be processed with sequence number 1, then the first exit distance range of channel W1A01 can be determined to include: the exit distance of the vehicle to be processed with sequence number 2; and between the vehicle to be processed with sequence number 2 and the exit, there are also vehicles to be processed with sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12.

[0066] The vehicle information corresponding to the vehicle to be processed with sequence number 2 is compared with the vehicle information corresponding to sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12. If the vehicle information is found to be consistent, then the vehicle to be processed with sequence number 2 is determined to be in the same batch as the vehicles to be processed with sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12. Therefore, the original vehicle to be processed with sequence number 2 can be replaced by the vehicle to be processed with sequence number 12. This can be understood as a normal situation. After outputting the vehicle body with sequence number 1, the vehicle body with sequence number 2 needs to be moved to the fine sorting area for output. However, since the vehicles with sequence number 2 and above are from the same batch, the vehicles closer to the exit can be output first, and then the vehicle body originally with sequence number 2 can be output last. At this point, the exit distance of the vehicle body with sequence number 2 is 1. This can also be understood as the exit distance of the vehicles with sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12 being 1 as well.

[0067] If inconsistencies are found in the vehicle information, it means that the vehicles to be processed are from different batches and cannot be offset. They need to be output sequentially according to their sequence numbers. In this case, the exit distance for the vehicle to be processed with sequence number 2 is 2.

[0068] S302. Based on multiple sheet-quantitative production plan data, obtain the second exit distance range of each storage channel when it is used as a receiving channel.

[0069] The second exit distance range includes the exit distance of each vehicle body to be processed on each storage channel.

[0070] Specifically, taking channel W1A01 as an example, channel W1A01 stores the sequence number 2 corresponding to a vehicle body to be processed. Then, when channel W1A01 is determined as the receiving channel, how many vehicles body to be processed need to be output before the vehicle body with sequence number 2 is output, and what are their corresponding sequence numbers?

[0071] Among them, channel W1A01 serves as the receiving channel. There are 19 storage channels between channel W1A01 and the output channel, namely channels W1A02-W1A03, channels W1A05-W1A07, channels W1B01-WB07, and channels W1C01-WC07. Among them, channels W1A02-W1A03 and channels W1A05-W1A07, as the senders, can send the vehicle body to be processed to channel W1A01 through channel W1A04.

[0072] If channel W1C01 stores multiple vehicles to be processed, with sequence numbers 12, 10, 8, and 6 respectively; channel W1C02 stores one vehicle to be processed with sequence number 5; channel W1C03 stores one vehicle to be processed with sequence number 3; channel W1C05 stores multiple vehicles to be processed with sequence numbers 11, 9, 7, and 4 respectively; channel W1C06 stores one vehicle to be processed with sequence number 1; channel W1A05 stores one vehicle to be processed with sequence number 71; and channel W1A06 stores one vehicle to be processed with sequence number 70, then the first exit distance range of channel W1A01 can be determined to include the exit distance of the vehicle to be processed with sequence number 2. At this time, there are also vehicles to be processed with sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 70, and 71 between the vehicle to be processed with sequence number 2 and the exit.

[0073] Similarly, compare the vehicle information corresponding to the vehicle to be processed with the vehicle information corresponding to the vehicle to be processed with the sequence numbers 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 70, and 71 to determine whether they are consistent and whether they can cancel each other out, thereby determining the exit distance of the vehicle to be processed with the sequence number 2.

[0074] S303. Based on multiple sheet-quantitative production plan data, obtain the distance range of the input channel when used as a sending channel.

[0075] The distance range of the input channel includes the exit distance of each vehicle body to be processed on the input channel.

[0076] S304. Based on multiple sheet-quantitative production plan data, obtain the range of the output channel's exit distance when used as a receiving channel.

[0077] The output channel's exit distance range includes the exit distance of each vehicle body to be processed on the output channel.

[0078] Since the input channel can only be used as a sending channel, the sequence number of all vehicles to be processed before each vehicle to be processed on the input channel is obtained.

[0079] Since the output channel can only be used as a receiving channel, the sequence number of all vehicles to be processed before each vehicle to be processed on the output channel is obtained.

[0080] In addition, the output order attributes of each storage channel also include: whether each storage channel is full of vehicles to be processed, the difference in exit distance between two adjacent vehicles to be processed on each storage channel, how many vehicles in each storage channel have a relative exit distance greater than the current vehicle to be processed from the previously stored vehicles to be processed, and the sequence number of each vehicle to be processed on each storage channel, totaling 8 attributes. Similarly, the output order attributes of the input channels and the output channels also include the above content.

[0081] The method provided in this application embodiment obtains the first exit distance range of each storage channel when used as a transmitting channel based on multiple tensor quantization production plan data; obtains the second exit distance range of each storage channel when used as a receiving channel based on multiple tensor quantization production plan data; obtains the distance range of the input channel when used as a transmitting channel based on multiple tensor quantization production plan data; and obtains the exit distance range of the output channel when used as a receiving channel based on multiple tensor quantization production plan data. The distance range between any channel and its exit in any state (transmit / receive) is calculated to generate instantaneous state attributes when constructing a reinforcement learning model.

[0082] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. The preset reinforcement learning model includes: Long Short Memory Network, Graph Neural Network, and Proximal Policy Optimization (PPO) algorithm. Figure 5 This is the fourth flowchart illustrating a reinforcement learning-based skid-type vehicle body storage area scheduling method provided in this application embodiment; as shown below. Figure 5 As shown, before generating the optimal scheduling order of the skid-type vehicle body storage area based on the instantaneous state attributes and using a preset reinforcement learning model, the method further includes: S401. Based on the positional relationship between the input channel, storage channel and output channel in the skid-type vehicle body storage area, construct the adjacency matrix of the skid-type vehicle body storage area.

[0083] In this embodiment, a set of 85 static, non-repeating point-to-point relationships of the input channel, storage channel, and output channel are stored and converted into numbers, such as [PIN,W1A01]->[0,1], which is used as an adjacency matrix of shape [2,85] for the graph neural network.

[0084] Based on the above, and using a pre-defined reinforcement learning model based on instantaneous state attributes, the optimal scheduling order for the skid-type vehicle storage area is generated, including: S402. Using a long short-term memory network, tensor compression is performed on the instantaneous state attributes to obtain the compressed state attributes.

[0085] All channels, except for the input and output channels, have two output order attributes: the output order attribute as the sender and the output order attribute as the receiver. In both states, two tensors of [21,7,8] dimensions are calculated using the distance to the output channel, and then concatenated along the last dimension to form a tensor of shape [21,7,16].

[0086] Calculate a [1,7,8] tensor and a [1,7,8]-dimensional all-zero tensor along the last dimension by using the input channel as the output order attribute of the sender. Calculate a [1,7,8]-dimensional all-zero tensor and the output channel as the output order attribute of the receiver by concatenating a [1,7,8]-dimensional tensor along the last dimension.

[0087] Then, the three tensors are concatenated along the first dimension in the order of input channel, storage channel, and output channel to obtain a tensor of dimension [23, 7, 16] as the instantaneous state attribute. Here, 23 means that the number of physical storage channels in the WBS is 21, plus the input channel and the output channel, which totals 23; 7 means that the physical structure of the WBS determines the maximum capacity, which is 7; and 16 means that the dimension for calculating the distance from the output is 8, which is the 8 attributes included in the output order attribute, and the total of the sender and receiver is 16.

[0088] Then, a long short-term memory network is used to perform tensor compression on the instantaneous state attribute [23,7,16] to obtain the compressed state attribute [23,64].

[0089] S403. Based on the compressed state attributes and the adjacency matrix, a graph neural network is used to transform the state attributes to obtain the transformed state attributes.

[0090] S404. Based on the transformed state attributes, the PPO algorithm is used to generate the optimal scheduling order for the skid-type vehicle body storage area.

[0091] Specifically, the compressed state attributes [23,64] and the adjacency matrix [2,85] are used as inputs to the graph neural network to obtain the transformed state attributes [1,2944], which are then used as inputs to the PPO algorithm to finally generate the optimal scheduling order for the skid-type vehicle body storage area.

[0092] The PPO algorithm is expressed as follows:

[0093] in, Let be the probability of taking action a in state s. Represented as a state-action value function, Let be the logarithmic probability of choosing action a in state s. Indicates the policy parameters The gradient of the cost function is obtained by taking the expectation of the product of the logarithmic probability and the state-action value function, and then taking the derivative with respect to the policy parameters.

[0094] Specifically, when training the preset reinforcement learning model, taking the skid-type vehicle body storage area as the WBS area as an example, firstly, a policy (actor) network and a value (critic) network are initialized and looped into the WBS queue. The transformed state attributes are passed into the policy network to output an action. The action is executed, the information in the WBS and the instantaneous reward are updated and saved. When the WBS queue is empty, the time difference error is calculated, the value network and policy network are updated, and the WBS queue is reloaded. The information in the WBS and the painting plan information are reset and looped into the WBS queue again until the loop number is reached, and finally the preset reinforcement learning model is obtained.

[0095] The system comprises nine actions: eight involving local agents and one without an agent. These actions determine the execution order of the eight transfer devices, or output an "no agent" message indicating that no action will be performed at this time. The reward system is as follows: For input channels, selecting the entrance agent and performing the "enter vehicle" action increases the reward; otherwise, the reward decreases as the waiting time increases. For output channels, selecting the exit agent and performing the "exit vehicle" action increases the reward; otherwise, the reward decreases as the waiting time increases. Selecting any agent with an action other than "no action" increases the reward; otherwise, a penalty is applied. Selecting the "no agent" when there is demand at the entrance or exit results in a penalty; otherwise, a low reward is applied.

[0096] The vehicle's daily entry and exit times from the Work Breakdown Structure (WBS) are queued in chronological order. Each value in the queue can only move to the next position after its processing is complete, effectively simulating real-world production conditions.

[0097] In the method provided in this application embodiment, an adjacency matrix of the skid-type vehicle body storage area is constructed based on the positional relationships between the input channels, storage channels, and output channels in the skid-type vehicle body storage area. A Long Short-Term Memory (LSTM) network is used to perform tensor compression on the instantaneous state attributes to obtain compressed state attributes. Based on the compressed state attributes and the adjacency matrix, a graph neural network is used for transformation to obtain transformed state attributes. Based on the transformed state attributes, the PPO algorithm is used to generate the optimal scheduling order of the skid-type vehicle body storage area. The overall state of the skid-type vehicle body storage area is calculated by combining the exit distance with the LSM network and the adjacency matrix, inputting them into the graph neural network, and transforming them into the state input of the PPO algorithm. Finally, the optimal scheduling order of the skid-type vehicle body storage area is obtained, which is the scheduling order of multiple target storage channels, and correspondingly, the optimal execution order of the target transfer equipment can be determined.

[0098] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Figure 6 This is the fifth flowchart illustrating a reinforcement learning-based method for scheduling storage areas in a skid-type vehicle body, as provided in this application embodiment; Figure 6 As shown, before generating the corresponding action instructions for the target mobile device based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels, the method further includes: S501. Obtain the action list of the target transfer machine. The action list of the target transfer machine contains multiple action information entries.

[0099] Each action message is used to instruct the transfer equipment to transport the vehicle body to be processed in a sending channel to a receiving channel.

[0100] Based on the above, according to the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels, corresponding action instructions for the target transfer device are generated, including: S502. Based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, the output order attributes of output channels, and the action list of the target transfer device, generate the corresponding action instructions for the target transfer device.

[0101] In this embodiment, since the target transfer device has multiple action information, one action information is determined as the action instruction of the target transfer device from the multiple action information based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels.

[0102] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Figure 7 A flowchart illustrating a reinforcement learning-based skid-type vehicle storage area scheduling method provided in this application is shown in Figure 6. Figure 7 As shown, before obtaining the action list of the target mobile device, the method further includes: S601. Based on the pick-up and drop-off range of each transfer machine within the skid-type vehicle storage area, the skid-type vehicle storage area is divided to obtain the movement area information of each transfer machine.

[0103] The mobile area information includes: information about the sending channel corresponding to each mobile device, and information about the receiving channel corresponding to each mobile device.

[0104] S602. Generate an action list for each transfer machine based on the movement zone information of each transfer machine.

[0105] In this embodiment, refer to Figure 1 Taking the transfer machine P02 as an example, the transfer range of the transfer machine P02 is divided into the skid-type vehicle body storage area to obtain the movement area information of the transfer machine P02. Among them, the information of the sending channel corresponding to the transfer machine P02 includes: input channel PIN and channel W1A04, and the information of the corresponding receiving channel includes: W1A05, W1A06 and W1A07.

[0106] The action list for transfer machine P02 is then determined, as shown in Table 1: Table 1. Action List of Transfer Machine Equipment P02

[0107] The action list of transfer machine P02 contains multiple action information entries. For example, PIN-W1A05 is one action information entry, and there are a total of 2×3+1 (empty actions), or 7 action information entries. Based on the movement area information of each transfer machine, the action list of each transfer machine can be generated.

[0108] In the method provided in this application embodiment, the skid-type vehicle storage area is divided according to the pick-up and drop-off range of each transfer machine within the skid-type vehicle storage area to obtain the movement area information of each transfer machine. The movement area information includes: the information of the corresponding sending channel and the information of the corresponding receiving channel for each transfer machine. Based on the movement area information of each transfer machine, an action list for each transfer machine is generated. The skid-type vehicle storage area is divided into 8 local intelligent agents, and the complex vehicle movement is transformed into the goal of each intelligent agent keeping the vehicle data within its jurisdiction within a controllable range by continuously receiving and sending data.

[0109] This application also provides another possible implementation of a reinforcement learning-based skid-type vehicle body storage area scheduling method. Wherein, Figure 8 This is the seventh flowchart illustrating a reinforcement learning-based skid-type vehicle storage area scheduling method provided in this application embodiment; as shown below. Figure 8 As shown, based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, the output order attributes of output channels, and the action list of the target transfer device, the corresponding action instructions for the target transfer device are generated, including: S701. If the first exit distance range of the sending channel corresponding to the target transfer device is not within the preset range of the first exit distance, then the first target sending channel is determined according to the output order attribute in the sending channel corresponding to the target transfer device.

[0110] Among them, the target transmission channel is the first transmission channel with the smallest exit distance range.

[0111] S702. Determine the first target receiving channel based on the output sequence attribute in the receiving channel corresponding to the target transfer device.

[0112] S703. Determine the action instructions of the target transfer machine based on the first target sending channel and the first target receiving channel.

[0113] In this embodiment, a concept area is introduced to manage the exit distance range. Specifically, the upper limit of the range for transfer machine devices P01 and P02 is 9999, the middle value is 9999, and the lower limit is 60; the upper limit of the range for transfer machine devices P03 and P04 is 9999, the middle value is 60, and the lower limit is 5; the upper limit of the range for transfer machine devices P05 and P06 is 60, the middle value is 25, and the lower limit is 1; the upper limit of the range for transfer machine device P07 is 25, the middle value is 1, and the lower limit is 1; and the upper limit of the range for transfer machine device P08 is 1, the middle value is 1, and the lower limit is 1. This can be understood as follows: if the exit distance of a vehicle body to be processed in the input channel is 20, then it is determined that the vehicle body to be processed is not within the control range of transfer equipment P01 and transfer equipment P02. In other words, the vehicle body to be processed is not stored in the receiving channel corresponding to transfer equipment P01 and transfer equipment P02, i.e., coarse sorting area 1 (A). It needs to be transmitted to coarse sorting area 2 (B) through the fast channel. Transfer equipment P03 and transfer equipment P04 then determine whether the vehicle body to be processed is within the control range of transfer equipment P03 and transfer equipment P04.

[0114] Specifically, taking the transfer device P05 as an example, it is determined whether the first exit distance range of the sending channel corresponding to the transfer device P05 is within the preset range of the first exit distance. The preset range of the first exit distance is greater than 1 and less than or equal to 60. If the first exit distance range of the sending channel corresponding to the transfer device P05 is not within the preset range of the first exit distance, then the first exit distance range of the sending channel corresponding to the transfer device P05 is less than or equal to 1 or greater than 60. Then it is determined whether there are multiple sending channels with the smallest first exit distance among the sending channels corresponding to the transfer device P05. If there are multiple sending channels with the smallest first exit distance, then the sending channel with the smallest sequence number among the multiple sending channels with the smallest first exit distance is determined as the first target sending channel.

[0115] For the transfer equipment P05, first count the number of negative distance differences between adjacent vehicle bodies in each receiving channel, and select the receiving channel with the smallest number. If there are multiple receiving channels that meet the conditions, further calculate the sum of the distance differences between adjacent vehicle bodies in each channel, where the minimum value is taken when the sum of the differences is greater than 0 and the maximum value is taken when it is less than 0. If there are still multiple receiving channels that meet the conditions, compare the difference in the painting plan sequence number between the first vehicle body in the sending channel and the last vehicle body in the receiving channel, and select the receiving channel with the smallest difference. If the differences are still the same, select the channel with the smallest channel number to determine the first target receiving channel. Finally, convert the index of the selected row by the number of senders and receivers to obtain the first target sending channel and the first target receiving channel as the action instructions for the transfer equipment P05.

[0116] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Figure 9 This is the eighth flowchart illustrating a reinforcement learning-based skid-type vehicle body storage area scheduling method provided in this application embodiment; as shown below. Figure 9 As shown, based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, the output order attributes of output channels, and the action list of the target transfer device, the corresponding action instructions for the target transfer device are generated, including: S801. If the first exit distance range of the sending channel corresponding to the target transfer device is within the first exit distance preset range, then determine whether the first exit distance range of the sending channel corresponding to the target transfer device is within the second exit distance preset range.

[0117] S802. If the distance range of the first exit in the sending channel corresponding to the target mobile device is determined to be within the preset range of the second exit distance, then it is determined whether the receiving channel corresponding to the target mobile device includes the return channel, and whether the sending channel and receiving channel corresponding to the target mobile device are fully loaded.

[0118] S803. If the receiving channel corresponding to the target transfer device includes a return channel, and the sending channel and receiving channel corresponding to the target transfer device are fully loaded, then the return channel is determined as the second target receiving channel, and the second target sending channel is determined according to the output order attribute in the sending channel corresponding to the target transfer device.

[0119] S804. Determine the action command of the target transfer machine based on the second target sending channel and the second target receiving channel.

[0120] In this embodiment, taking the transfer device P05 as an example, if the first exit distance of the sending channel corresponding to the transfer device P05 is within a preset range of the first exit distance, then the first exit distance range of the sending channel corresponding to the transfer device P05 is greater than 1 and less than or equal to 60. Then it is determined whether the sending channel corresponding to the transfer device P05 is within a preset range of the second exit distance, which is greater than 25 and less than or equal to 60.

[0121] If it is determined that the first exit distance in the transmitting channel corresponding to the transfer device P05 is within the preset range of the second exit distance, then it is determined whether the receiving channel corresponding to the transfer device P05 includes the return channel, and whether the transmitting and receiving channels corresponding to the transfer device P05 are fully loaded. If it is determined that the receiving channel corresponding to the transfer device P05 includes the return channel, i.e., channel W1B04, and both the transmitting channels (W1B01, W1B02, W1B03, W1C04) and receiving channels (W1C01, W1C02, W1C03) corresponding to the transfer device P05 are fully loaded, then the return channel W1B04 is determined to be the second target receiving channel.

[0122] If there is a distance difference among the sending channels (W1B01, W1B02, W1B03, W1C04) corresponding to the transfer machine equipment P05, the sending channel with the closest exit distance is determined as the second target sending channel based on the distance difference. Finally, the second target sending channel and the second target receiving channel are used as the action command for the transfer machine equipment P05.

[0123] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Figure 10 A flowchart illustrating a reinforcement learning-based skid-type vehicle body storage area scheduling method provided in this application embodiment is shown in Figure 9. Figure 10 As shown, the method also includes: S901, Obtain the execution status of the action command.

[0124] S902. If the execution status of the action command is not executed, then determine the operating status of the target transfer machine.

[0125] S903. If the target transfer device is in an idle state, then the action command is sent to the target transfer device, the execution status of the action command is updated to the executed state, and the working status of the target transfer device is updated to the running state.

[0126] S904. If the working state of the target transfer machine changes to the idle state, it is determined that the target transfer machine has completed the execution, and the execution status of the action command is updated to the execution completed state.

[0127] In this embodiment, the instructions issued by the scheduling system generally include multiple movement instructions, which need to be broken down and sent to the corresponding target transfer equipment for execution. After being broken down, the state of each individual action instruction is initially set to an inactive state. For example, ["W1C01:P07:POUT", "W1B01:P05:W1C01", "W1A02:P03:W1B01"] is a set of instructions issued by the scheduling system. "W1C01:P07:POUT" is a single instruction, meaning: transfer machine P07 goes to lane W1C01, picks up the first vehicle, and then delivers it to POUT; W1A02:P03:W1B01 means that transfer machine P03 goes from lane W1A02, picks up the first vehicle, and then delivers it to lane W1B01.

[0128] If the initial state of a single action instruction is not executed, read the signal bit of the programmable logic controller (PLC) of the target mobile machine hardware. If the target mobile machine is in an idle state, send the instruction to the instruction receiving address of the PLC of the target mobile machine. After sending, update the current instruction state to executed.

[0129] When the action command is in the executed state, the value of the electronic tag (RFID) on the target transfer machine is queried repeatedly. If the value read is not empty, the RFID information of the command is updated, and the vehicle's unique code information is extracted from the RFID information. At the same time, the operating status of the target transfer machine corresponding to the action command is checked. If the target transfer machine is in the idle state, it means that the target transfer machine has delivered the vehicle to the designated location. At this time, the dynamic command status is updated to the execution completed state.

[0130] When the action command status is "execution completed", the vehicle's unique code information read from the action command is retrieved, the vehicle's position information is updated, and after the update is complete, the action command status is updated to "closed".

[0131] Completing the above steps signifies the completion of an action instruction. All action instructions are independently managed according to these steps. When all action instructions in an instruction set are in a disabled state, it indicates the completion of a scheduling request, and then the next round of instruction sets is requested and executed.

[0132] This application also provides another possible implementation of a skid-type vehicle body storage area scheduling method based on reinforcement learning. Figure 11 A flowchart illustrating a reinforcement learning-based skid-type vehicle storage area scheduling method provided in this application is shown in Figure 10. Figure 11 As shown, the method also includes: S1001. Obtain the preset empty skid plan information.

[0133] S1002. Based on the preset empty skid plan information, sort the empty skids to be transferred in the skid-type vehicle body storage area to obtain the transfer order of each empty skid to be transferred.

[0134] S1003. Schedule the transfer of each empty skid to be transferred according to the transfer order.

[0135] In this embodiment, the skid is a carrier for the car body to be processed. The transfer machine outputs the car body on the skid by moving the skid. For example, if the transfer machine in the body-in-white storage area outputs a car body to be processed to the painting workshop, the skid corresponding to the car body to be processed needs to be transferred to the welding workshop through the skid transfer channel. However, if the skid transfer channel is full, in order to avoid congestion, the skid needs to be transferred from the return channel to the body-in-white storage area to temporarily store the skid and wait for the instruction to output it again. At this time, there is no car body to be processed on the skid, so it is an empty skid.

[0136] The output process for empty skids mainly involves acquiring preset empty skid plan information, adding a unique code to the empty skid, and converting the location information of the empty skids in the skid-type body storage area into a painting plan. By default, all empty skids in the skid-type body storage area are in a suspended state. The original date and planned date are both 9999-12-30, and the original sequence and planned sequence are both YY001. Before being transmitted to the scheduling system, the preset empty skid plan information and the empty skid information in the skid-type body storage area are matched according to the skid location from fine-grained to coarse-grained, clarifying the transfer order of the empty skids in the overall production plan. Finally, the transfer scheduling of each empty skid to be transferred is performed according to the transfer order.

[0137] In the method provided in this application embodiment, the preset empty skid plan and the empty skid information in the skid-type vehicle body storage area are stored separately. The preset empty skid plan and the empty skid information are only bound together when forming an empty skid transfer, which is highly flexible and reduces workshop learning costs.

[0138] The following will continue to explain the skid-type vehicle body storage area scheduling device and electronic device based on reinforcement learning provided in any of the above embodiments of this application. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, the parts not mentioned in this embodiment can be referred to the corresponding content in the method embodiment.

[0139] Figure 12 This is a schematic diagram of the functional modules of a skid-type vehicle body storage area scheduling device based on reinforcement learning, provided in an embodiment of this application. Figure 12 As shown, the reinforcement learning-based skid-type vehicle storage area scheduling device 100 includes: The acquisition module 110 is used to acquire the real-time production plan of the car body production line. The real-time production plan includes multiple production plan data corresponding to multiple cars to be processed. The processing module 120 is used to perform tensor quantization processing on multiple production plan data to obtain corresponding multiple tensor quantized production plan data; each tensor quantized production plan data includes: body information and sequence number of multiple vehicles to be processed; The determination module 130 is used to determine the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type body storage area based on multiple tensile production plan data. Module 140 is used to construct the instantaneous state attributes of the skid-type body storage area based on the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel. The generation module 150 is used to generate the optimal scheduling order of the skid-type vehicle body storage area based on the instantaneous state attributes and using a preset reinforcement learning model. The optimal scheduling order includes the scheduling order of multiple target storage channels. The generation module 150 is used to generate corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of multiple target storage channels, the output order attributes of input channels, and the output order attributes of output channels.

[0140] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0141] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0142] Figure 13 This is a schematic diagram of an electronic device provided in an embodiment of this application. This electronic device can be used for reinforcement learning-based skid-type vehicle storage area scheduling. Figure 13 As shown, the electronic device includes: a processor 210, a storage medium 220, and a bus 230.

[0143] Storage medium 220 stores machine-readable instructions executable by processor 210. When the electronic device is running, processor 210 communicates with storage medium 220 via bus 230, and processor 210 executes the machine-readable instructions to perform the steps of the above method embodiment. The specific implementation and technical effects are similar and will not be described again here.

[0144] Optionally, this application also provides a storage medium 220, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the above-described method embodiments. The specific implementation and technical effects are similar, and will not be repeated here.

[0145] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A skid-type vehicle storage area scheduling method based on reinforcement learning, characterized in that, The method includes: Obtain the real-time production plan of the vehicle body production line, wherein the real-time production plan includes: multiple production plan data corresponding to multiple vehicles to be processed; Tensorization processing is performed on the multiple production plan data to obtain corresponding multiple tensorized production plan data; each tensorized production plan data includes: the body information and sequence number of the multiple vehicles to be processed; Based on the multiple production plan data, the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type body storage area are determined respectively. Based on the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel, the instantaneous state attributes of the skid-type vehicle body storage area are constructed. Based on the instantaneous state attributes, a preset reinforcement learning model is used to generate the optimal scheduling order of the skid-type vehicle body storage area. The optimal scheduling order includes the scheduling order of multiple target storage channels. Based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels, corresponding action instructions for the target transfer device are generated.

2. The method according to claim 1, characterized in that, Each production plan data includes: original part number, original color design, original status code, original material version, original date, original batch sequence, and sequence information; the body information of the multiple vehicles to be processed includes: quantified part number, quantified color design, quantified status code, quantified material version, quantified original date, quantified original batch, and quantified original single-unit sequence; The tensor quantization process of the multiple production plan data to obtain corresponding tensor quantized production plan data includes: Perform hash modulo operations on the original part number and the original color design respectively to obtain the quantized part number and the quantized color design; The original state code is truncated to obtain the tensor state code; The original material version and the original date are respectively converted into numerical values ​​to obtain the quantified material version and the quantified original date. The original batch sequence is decomposed to obtain the tensor-quantized original batch and the tensor-quantized original single-unit sequence; The sequence information is converted into a sequence number to obtain the sequence number.

3. The method according to claim 1, characterized in that, The output order attributes of each storage channel include: a first exit distance range of each storage channel when it is a transmitting channel, and a second exit distance range of each storage channel when it is a receiving channel; the output order attributes of the input channel include: the distance range of the input channel when it is a transmitting channel; and the output order attributes of the output channel include: the exit distance range of the output channel when it is a receiving channel. The step of determining the output order attributes of each storage channel, the output order attributes of the input channel, and the output order attributes of the output channel in the skid-type vehicle body storage area based on the multiple tensile production plan data includes: Based on the multiple production plan data, the first exit distance range of each storage channel when used as a sending channel is obtained. The first exit distance range includes the exit distance of each vehicle body to be processed on each storage channel. Based on the multiple production plan data, the second exit distance range of each storage channel when used as a receiving channel is obtained. The second exit distance range includes the exit distance of each vehicle body to be processed on each storage channel. Based on the multiple production plan data, the distance range of the input channel when it is used as the sending channel is obtained. The distance range of the input channel includes the exit distance of each vehicle body to be processed on the input channel. Based on the multiple production plan data, the exit distance range of the output channel when it is used as a receiving channel is obtained. The exit distance range of the output channel includes the exit distance of each vehicle body to be processed on the output channel.

4. The method according to claim 1, characterized in that, The preset reinforcement learning model includes: Long Short-Term Memory Network, Graph Neural Network, and Proximal Policy Optimization (PPO) algorithm; Before generating the optimal scheduling order of the skid-type vehicle body storage area using a preset reinforcement learning model based on the instantaneous state attributes, the method further includes: Based on the positional relationship between the input channel, storage channel, and output channel in the skid-type vehicle body storage area, an adjacency matrix of the skid-type vehicle body storage area is constructed. The step of generating the optimal scheduling order of the skid-type vehicle body storage area based on the instantaneous state attributes and using a preset reinforcement learning model includes: The instantaneous state attributes are tensor-compressed using the Long Short-Term Memory network to obtain compressed state attributes. Based on the compressed state attributes and the adjacency matrix, the graph neural network is used to perform the transformation to obtain the transformed state attributes; Based on the transformed state attributes, the PPO algorithm is used to generate the optimal scheduling order for the skid-type vehicle body storage area.

5. The method according to claim 1, characterized in that, Before generating the corresponding action command for the target mobile device based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels, the method further includes: Obtain the action list of the target transfer machine device. The action list of the target transfer machine device records multiple action information. Each action information is used to instruct the transfer machine device to transport the vehicle body to be processed in a sending channel to a receiving channel. The step of generating corresponding action instructions for the target transfer device based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, and the output order attributes of the output channels includes: Based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer device, the corresponding action instructions for the target transfer device are generated.

6. The method according to claim 5, characterized in that, Before obtaining the action list of the target mobile device, the method further includes: Based on the delivery range of each transfer machine within the skid-type vehicle storage area, the skid-type vehicle storage area is divided to obtain the movement area information of each transfer machine. The movement area information includes: the information of the sending channel corresponding to each transfer machine and the information of the receiving channel corresponding to each transfer machine. Based on the movement area information of each transfer machine, an action list for each transfer machine is generated.

7. The method according to claim 6, characterized in that, The step of generating corresponding action instructions for the target transfer machine based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer machine includes: If the first exit distance range of the sending channel corresponding to the target transfer device is not within the preset range of the first exit distance, then the first target sending channel is determined according to the output order attribute in the sending channel corresponding to the target transfer device; wherein, the target sending channel is the first sending channel with the smallest exit distance range; The first target receiving channel is determined based on the output order attribute in the receiving channel corresponding to the target transfer device; The action instructions for the target transfer device are determined based on the first target sending channel and the first target receiving channel.

8. The method according to claim 7, characterized in that, The step of generating corresponding action instructions for the target transfer machine based on the optimal scheduling order, the output order attributes of the multiple target storage channels, the output order attributes of the input channels, the output order attributes of the output channels, and the action list of the target transfer machine further includes: If the first exit distance of the sending channel corresponding to the target mobile device is within the first exit distance preset range, then it is determined whether the first exit distance range of the sending channel corresponding to the target mobile device is within the second exit distance preset range. If it is determined that the first exit distance in the sending channel corresponding to the target mobile device is within the second exit distance preset range, then it is determined whether the receiving channel corresponding to the target mobile device includes a return channel, and whether the sending channel and receiving channel corresponding to the target mobile device are fully loaded. If the receiving channel corresponding to the target mobile device includes the return channel, and the sending and receiving channels corresponding to the target mobile device are fully loaded, then the return channel is determined as the second target receiving channel; and the second target sending channel is determined according to the output order attribute in the sending channel corresponding to the target mobile device. The action instructions for the target transfer device are determined based on the second target sending channel and the second target receiving channel.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the execution status of the action command; If the execution status of the action command is in the non-execution state, then the operating status of the target mobile device is determined; If the target mobile device is in an idle state, the action command is sent to the target mobile device, the execution status of the action command is updated to executed, and the working status of the target mobile device is updated to running. If the working state of the target transfer device changes to an idle state, it is determined that the target transfer device has completed its execution, and the execution status of the action command is updated to the completed state.

10. The method according to claim 1, characterized in that, The method further includes: Obtain the preset empty skid plan information; According to the preset empty skid plan information, the empty skids to be transferred in the skid-type vehicle body storage area are sorted to obtain the transfer order of the empty skids to be transferred. The empty sleds to be transferred are scheduled according to the transfer order.

Images