Knowledge distillation-based refrigerating capacity prediction method and device

Abstract

The invention relates to the technical field of refrigerating capacity prediction, and provides a refrigerating capacity prediction method and device based on knowledge distillation. The method comprises the following steps: constructing a feature extraction network by using a multi-head self-attention network, a feedforward neural network and a full connection layer, constructing a feature processing network by using the full connection layer and a Transform module, and constructing a teacher model and a student model by using the feature extraction network and the feature processing network; acquiring first historical data of the first place and second historical data of the second place; performing refrigerating capacity prediction training on the teacher model by using the first historical data; performing refrigerating capacity prediction training on the student model by using the second historical data, and constraining the student model by using the teacher model in the process of performing the refrigerating capacity prediction training on the student model; and utilizing the student model to predict the refrigerating capacity required by the second place at different times every day.

Description

Technical Field

[0001] The present application relates to the technical field of refrigeration capacity prediction, and in particular to a refrigeration capacity prediction method and device based on knowledge distillation. Background Art

[0002] Current cooling capacity prediction methods are mainly based on machine learning and statistical models, which usually require a large amount of historical data to train the models, such as support vector machines, decision trees, neural networks, etc. The core of these models is to predict future cooling demand by learning patterns in historical data. For example, in a shopping mall with sufficient data, data based on characteristics such as ambient temperature, humidity, and traffic flow can accurately predict cooling demand. However, when the amount of training data is small, the model is prone to overfitting and has poor generalization ability. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a refrigeration capacity prediction method, device, electronic device and computer-readable storage medium based on knowledge distillation to solve the problem of overfitting and poor generalization ability of refrigeration capacity prediction caused by small amount of training data in the prior art.

[0004] According to a first aspect of an embodiment of the present application, a method for predicting refrigeration capacity based on knowledge distillation is provided, comprising: constructing a feature extraction network using a multi-head self-attention network, a feedforward neural network and a fully connected layer, constructing a feature processing network using a fully connected layer and a Transformer module, and constructing a teacher model and a student model using the feature extraction network and the feature processing network, wherein the network structures of the teacher model and the student model are the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; obtaining first historical data of a first place and second historical data of a second place, wherein the first historical data and the second historical data both include: place data, environmental data, weather data and refrigeration capacity; using the first historical data to train the teacher model for refrigeration capacity prediction; using the second historical data to train the student model for refrigeration capacity prediction, and in the process of training the student model for refrigeration capacity prediction, using the teacher model to constrain the student model; and using the student model to predict the refrigeration capacity required for the second place at different times of the day.

[0005] According to a second aspect of an embodiment of the present application, a refrigeration capacity prediction device based on knowledge distillation is provided, comprising: a construction module, configured to construct a feature extraction network using a multi-head self-attention network, a feedforward neural network and a fully connected layer, construct a feature processing network using a fully connected layer and a Transformer module, and construct a teacher model and a student model using the feature extraction network and the feature processing network, wherein the network structure of the teacher model and the student model is the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; an acquisition module, configured to obtain first historical data of a first place and second historical data of a second place, wherein the first historical data and the second historical data both include: place data, environmental data, weather data and refrigeration capacity; a first training module, configured to perform refrigeration capacity prediction training on the teacher model using the first historical data; a second training module, configured to perform refrigeration capacity prediction training on the student model using the second historical data, and in the process of performing refrigeration capacity prediction training on the student model, use the teacher model to constrain the student model; a prediction module, configured to use the student model to predict the refrigeration capacity required for the second place at different times of the day.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: constructing a feature extraction network using a multi-head self-attention network, a feedforward neural network and a fully connected layer, constructing a feature processing network using a fully connected layer and a Transformer module, and constructing a teacher model and a student model using a feature extraction network and a feature processing network, wherein the network structures of the teacher model and the student model are the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; obtaining the first historical data of the first place and the second historical data of the second place, wherein the first historical data and the second historical data both include: place data, environmental data, weather data and cooling capacity; using the first historical data to train the teacher model for cooling capacity prediction; using the second historical data to train the student model for cooling capacity prediction, and in the process of training the student model for cooling capacity prediction, using the teacher model to constrain the student model; using the student model to predict the cooling capacity required for the second place at different times of the day. The above technical means can solve the problem of overfitting and poor generalization ability of cooling capacity prediction caused by small amount of training data in the prior art, thereby avoiding overfitting of cooling capacity prediction and improving generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flow chart of a refrigeration capacity prediction method based on knowledge distillation provided in an embodiment of the present application; Figure 2 It is a flow chart of a cooling capacity prediction method provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of a refrigeration capacity prediction device based on knowledge distillation provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0012] A refrigeration capacity prediction method and device based on knowledge distillation according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0013] Figure 1 It is a flow chart of a refrigeration capacity prediction method based on knowledge distillation provided in an embodiment of the present application. Figure 1 The refrigeration capacity prediction method based on knowledge distillation can be executed by a computer or a server, or by software on the computer or the server. Figure 1 As shown, the refrigeration capacity prediction method based on knowledge distillation includes: S101, using a multi-head self-attention network, a feedforward neural network and a fully connected layer to build a feature extraction network, using a fully connected layer and a Transformer module to build a feature processing network, and using the feature extraction network and the feature processing network to build a teacher model and a student model, wherein the network structure of the teacher model and the student model is the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; S102, obtaining first historical data of a first location and second historical data of a second location, wherein the first historical data and the second historical data both include: location data, environment data, weather data, and cooling capacity; S103, using the first historical data to perform cooling capacity prediction training on the teacher model; S104, using the second historical data to perform cooling capacity prediction training on the student model, and in the process of performing cooling capacity prediction training on the student model, using the teacher model to constrain the student model; S105, using the student model to predict the cooling capacity required by the second location at different times of the day.

[0014] The first place has multiple first historical data, and the second place has multiple second historical data, and the number of first historical data far exceeds the number of second historical data. The first place has abundant data, and the second place has less data. If the data of the second place is used to train the model applied to the second place, it will lead to problems of overfitting and poor generalization ability. In order to solve this problem, the embodiment of the present application first uses the first historical data of the first place to train the teacher model, and then based on the principle of knowledge distillation, in the process of using the second historical data to train the student model for refrigeration capacity prediction, the teacher model is used to constrain the student model. The student model obtained in this way can be well applied in the second place.

[0015] The venue data includes the latitude and longitude of the venue, the floor area, the total building area of ​​the venue, the total number of shops in the venue, the distribution and proportion of the venue business types (the distribution and proportion of different types of shops). The environmental data includes the temperature, humidity, and flow of people inside the venue. The weather data includes the temperature, humidity, and wind speed outside the venue. The cooling capacity is the actual cooling capacity used in the venue data, environmental data, and weather data. It is used as a label in subsequent training.

[0016] The network structures of the teacher model and the student model are the same, which means that both the teacher model and the student model contain a feature extraction network and a feature processing network, and the number and order of network layers contained in the feature extraction network or the feature processing network in the teacher model and the student model are the same. The difference is that the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model.

[0017] According to the technical solution provided by the embodiment of the present application, a feature extraction network is constructed using a multi-head self-attention network, a feedforward neural network and a fully connected layer, a feature processing network is constructed using a fully connected layer and a Transformer module, and a teacher model and a student model are constructed using the feature extraction network and the feature processing network, wherein the network structure of the teacher model and the student model is the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; the first historical data of the first place and the second historical data of the second place are obtained, wherein the first historical data and the second historical data both include: place data, environmental data, weather data and cooling capacity; the teacher model is trained for cooling capacity prediction using the first historical data; the student model is trained for cooling capacity prediction using the second historical data, and in the process of training the student model for cooling capacity prediction, the teacher model is used to constrain the student model; the student model is used to predict the cooling capacity required for the second place at different times of the day. The above technical means can be used to solve the problem of overfitting and poor generalization ability of cooling capacity prediction caused by small amount of training data in the prior art, thereby avoiding overfitting of cooling capacity prediction and improving generalization ability.

[0018] In an optional embodiment, historical data of the first shopping mall is collected, including the mall's own dimensional data, shopping mall environmental data, outdoor weather data, date data, and corresponding hourly cooling data, etc. The discrete features are converted into category types and then converted into numerical values ​​for embedding, and the numerical features are normalized. The data set is divided in chronological order, including a training data set, a test data set, and a validation data set. The network structure of the automatic encoding and decoding network is defined, the optimizer is determined, the learning rate, the number of iterations, and other model training are determined, and finally the teacher model is trained using the first historical data, and the optimal model is selected and saved. Later, in the process of using the second historical data to train the student model for cooling capacity prediction training, the teacher model is used to constrain the student model. Training the student model using the second historical data is similar to training the teacher model using the first historical data.

[0019] Furthermore, a feature extraction network is constructed using a multi-head self-attention network, a feedforward neural network and a fully connected layer, including: sequentially connecting the multi-head self-attention network, the first residual connection, the first batch normalization layer, the feedforward neural network, the first fully connected layer, the activation layer, the second fully connected layer, the second residual connection and the second batch normalization layer to obtain a feature extraction network; wherein the first residual connection is used to connect the input end of the first batch normalization layer with the input end and the output end of the multi-head self-attention network, and the second residual connection is used to connect the input end of the second batch normalization layer with the output end of the second fully connected layer and the input end of the multi-head self-attention network.

[0020] The multi-head self-attention network, residual connection, batch normalization layer, feedforward neural network, fully connected layer, activation layer, fully connected layer, residual connection and batch normalization layer are connected in sequence to obtain a feature extraction network. The "first" and "second" in the above text are only used to distinguish the order. The first residual connection actually adds the input and output of the multi-head self-attention network as the input of the first batch normalization layer, and the second residual connection actually adds the input of the multi-head self-attention network and the output of the second fully connected layer as the input of the second batch normalization layer.

[0021] The number of dimensions of the fully connected layers in the student model is less than that in the teacher model. For example, the first and second fully connected layers in the feature extraction network of the teacher model are 256- and 128-dimensional networks (the first fully connected layer outputs 256-dimensional features, and the second fully connected layer outputs 128-dimensional features), while the first and second fully connected layers in the feature extraction network of the student model are 64- and 32-dimensional networks (the first fully connected layer outputs 64-dimensional features, and the second fully connected layer outputs 32-dimensional features).

[0022] Furthermore, a feature processing network is constructed using a fully connected layer and a Transformer module, including: sequentially connecting an embedding layer, a fully connected layer, a batch normalization layer, an activation layer, a Transformer module, a fully connected layer, a batch normalization layer, an activation layer, a fully connected layer, a batch normalization layer, and an activation layer to obtain a feature processing network.

[0023] The Transformer module is a deep learning architecture widely used in natural language processing (NLP) tasks. It was first proposed by Vaswani et al. in their 2017 paper "Attention is All You Need". The Transformer model replaces the traditional recurrent neural network (RNN) and convolutional neural network (CNN) with a self-attention mechanism, and performs well in processing long dependencies and parallel training. The other network layers are all existing network layers and will not be described in detail.

[0024] The number of dimensions of the fully connected layers in the student model is less than that in the teacher model. For example, the first fully connected layer, the second fully connected layer, and the third fully connected layer in the feature processing network of the teacher model are 256-, 128-, and 64-dimensional networks (the first fully connected layer outputs 256-dimensional features, the second fully connected layer outputs 128-dimensional features, and the third fully connected layer outputs 64-dimensional features), while the first fully connected layer, the second fully connected layer, and the third fully connected layer in the feature processing network of the student model are 64-, 32-, and 16-dimensional networks (the first fully connected layer outputs 64-dimensional features, the second fully connected layer outputs 32-dimensional features, and the third fully connected layer outputs 16-dimensional features).

[0025] Furthermore, the teacher model is trained on cooling capacity prediction using the first historical data, including: inputting the location data, environment data and weather data in the first historical data into the teacher model, and within the teacher model: processing the location data, environment data and weather data in the first historical data respectively through a feature extraction network to obtain location characteristics, environment characteristics and weather characteristics; processing the location characteristics, environment characteristics and weather characteristics through a feature processing network to obtain a first prediction value corresponding to the first historical data; calculating a first prediction loss between the first prediction value and the cooling capacity in the first historical data using a regression loss function; and optimizing the weights in the teacher model based on the first prediction loss.

[0026] The site features, environmental features and weather features may be spliced, and the splicing result may be input into a feature processing network, and the feature processing network may determine a first prediction value corresponding to the first historical data based on the splicing result. The regression loss function may use Smooth L1 Loss.

[0027] Furthermore, the student model is trained on cooling capacity prediction using the second historical data, and in the process of training the student model on cooling capacity prediction, the teacher model is used to constrain the student model, including: inputting the location data, environmental data and weather data in the second historical data into the teacher model, and outputting a second prediction value corresponding to the second historical data; inputting the location data, environmental data and weather data in the second historical data into the student model, and outputting a third prediction value corresponding to the second historical data; calculating the constraint loss between the second prediction value and the third prediction value using a regression loss function; calculating the second prediction loss between the third prediction value and the cooling capacity in the second historical data using a regression loss function; and optimizing the weights in the student model based on the second prediction loss and the constraint loss.

[0028] Furthermore, the place data, environment data and weather data in the second historical data are input into the teacher model, and a second prediction value corresponding to the second historical data is output, including: inputting the place data, environment data and weather data in the second historical data into the teacher model, and within the teacher model: processing the place data, environment data and weather data in the second historical data respectively through a feature extraction network to obtain place characteristics, environment characteristics and weather characteristics; processing the place characteristics, environment characteristics and weather characteristics through a feature processing network to obtain a second prediction value corresponding to the second historical data.

[0029] Furthermore, the place data, environment data and weather data in the second historical data are input into the student model, and a third prediction value corresponding to the second historical data is output, including: inputting the place data, environment data and weather data in the second historical data into the student model, and within the student model: processing the place data, environment data and weather data in the second historical data respectively through a feature extraction network to obtain place characteristics, environment characteristics and weather characteristics; processing the place characteristics, environment characteristics and weather characteristics through a feature processing network to obtain a third prediction value corresponding to the second historical data.

[0030] Further, optimizing the weights in the student model according to the second prediction loss and the constraint loss includes: calculating the total loss L according to the following formula, and optimizing the weights in the student model according to the total loss: L=alpha * L1 + (1 - alpha) *L2; Among them, alpha is a preset hyperparameter, the range of alpha is between 0 and 1. The larger the aphha is, the more knowledge the student model uses the teacher model. L1 is the constraint loss, and L2 is the second prediction loss.

[0031] Figure 2 is a flow chart of a cooling capacity prediction method provided in an embodiment of the present application. Figure 2As shown, the method includes: S201, obtaining location data, environment data and weather data of a second location at the current moment; S202, inputting the location data, environment data and weather data of the second location at the current moment into the student model, and inside the student model: S203, respectively processing the location data, environment data and weather data of the second location at the current moment through a feature extraction network to obtain location features, environment features and weather features; S204, processing the site characteristics, environmental characteristics and weather characteristics through the feature processing network to obtain the cooling capacity required by the second site at the current moment.

[0032] The embodiment of the present application is the reasoning stage of the student model. Prior to the embodiment of the present application, the reasoning stage and the training stage of the network correspond to each other, and the reasoning stage will not be explained in detail here.

[0033] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0034] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0035] Figure 3 Schematic diagram of a refrigeration capacity prediction device based on knowledge distillation provided in an embodiment of the present application. Figure 3 As shown, the refrigeration capacity prediction device based on knowledge distillation includes: A construction module 301 is configured to construct a feature extraction network using a multi-head self-attention network, a feedforward neural network, and a fully connected layer, construct a feature processing network using a fully connected layer and a Transformer module, and construct a teacher model and a student model using the feature extraction network and the feature processing network, wherein the network structures of the teacher model and the student model are the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; An acquisition module 302 is configured to acquire first historical data of a first location and second historical data of a second location, wherein the first historical data and the second historical data both include: location data, environment data, weather data and cooling capacity; A first training module 303 is configured to perform cooling capacity prediction training on the teacher model using the first historical data; The second training module 304 is configured to perform cooling capacity prediction training on the student model using the second historical data, and in the process of performing cooling capacity prediction training on the student model, constrain the student model using the teacher model; The prediction module 305 is configured to use the student model to predict the cooling capacity required by the second location at different times of the day.

[0036] According to the technical solution provided by the embodiment of the present application, a feature extraction network is constructed using a multi-head self-attention network, a feedforward neural network and a fully connected layer, a feature processing network is constructed using a fully connected layer and a Transformer module, and a teacher model and a student model are constructed using the feature extraction network and the feature processing network, wherein the network structure of the teacher model and the student model is the same, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; the first historical data of the first place and the second historical data of the second place are obtained, wherein the first historical data and the second historical data both include: place data, environmental data, weather data and cooling capacity; the teacher model is trained for cooling capacity prediction using the first historical data; the student model is trained for cooling capacity prediction using the second historical data, and in the process of training the student model for cooling capacity prediction, the teacher model is used to constrain the student model; the student model is used to predict the cooling capacity required for the second place at different times of the day. The above technical means can be used to solve the problem of overfitting and poor generalization ability of cooling capacity prediction caused by small amount of training data in the prior art, thereby avoiding overfitting of cooling capacity prediction and improving generalization ability.

[0037] In an optional embodiment, historical data of the first shopping mall is collected, including the mall's own dimensional data, shopping mall environmental data, outdoor weather data, date data, and corresponding hourly cooling data, etc. The discrete features are converted into category types and then converted into numerical values ​​for embedding, and the numerical features are normalized. The data set is divided in chronological order, including a training data set, a test data set, and a validation data set. The network structure of the automatic encoding and decoding network is defined, the optimizer is determined, the learning rate, the number of iterations, and other model training are determined, and finally the teacher model is trained using the first historical data, and the optimal model is selected and saved. Later, in the process of using the second historical data to train the student model for cooling capacity prediction training, the teacher model is used to constrain the student model. Training the student model using the second historical data is similar to training the teacher model using the first historical data.

[0038] In some embodiments, the construction module 301 is also configured to sequentially connect a multi-head self-attention network, a first residual connection, a first batch normalization layer, a feedforward neural network, a first fully connected layer, an activation layer, a second fully connected layer, a second residual connection, and a second batch normalization layer to obtain a feature extraction network; wherein the first residual connection is used to connect the input end of the first batch normalization layer with the input end and output end of the multi-head self-attention network, and the second residual connection is used to connect the input end of the second batch normalization layer with the output end of the second fully connected layer and the input end of the multi-head self-attention network.

[0039] In some embodiments, the construction module 301 is also configured to sequentially connect the embedding layer, the fully connected layer, the batch normalization layer, the activation layer, the Transformer module, the fully connected layer, the batch normalization layer, the activation layer, the fully connected layer, the batch normalization layer and the activation layer to obtain a feature processing network.

[0040] In some embodiments, the first training module 303 is also configured to input the place data, environment data and weather data in the first historical data into the teacher model, and within the teacher model: the place data, environment data and weather data in the first historical data are processed separately by a feature extraction network to obtain place characteristics, environment characteristics and weather characteristics; the place characteristics, environment characteristics and weather characteristics are processed by a feature processing network to obtain a first prediction value corresponding to the first historical data; the regression loss function is used to calculate the first prediction loss between the first prediction value and the cooling capacity in the first historical data; and the weights in the teacher model are optimized based on the first prediction loss.

[0041] In some embodiments, the second training module 304 is also configured to input the location data, environmental data and weather data in the second historical data into the teacher model, and output a second prediction value corresponding to the second historical data; input the location data, environmental data and weather data in the second historical data into the student model, and output a third prediction value corresponding to the second historical data; calculate the constraint loss between the second prediction value and the third prediction value using a regression loss function; calculate the second prediction loss between the third prediction value and the cooling capacity in the second historical data using a regression loss function; and optimize the weights in the student model based on the second prediction loss and the constraint loss.

[0042] In some embodiments, the second training module 304 is also configured to input the place data, environment data and weather data in the second historical data into the teacher model, and within the teacher model: the place data, environment data and weather data in the second historical data are processed separately by a feature extraction network to obtain place characteristics, environment characteristics and weather characteristics; the place characteristics, environment characteristics and weather characteristics are processed by a feature processing network to obtain a second prediction value corresponding to the second historical data.

[0043] In some embodiments, the second training module 304 is also configured to input the place data, environment data and weather data in the second historical data into the student model, and within the student model: the place data, environment data and weather data in the second historical data are processed separately by a feature extraction network to obtain place characteristics, environment characteristics and weather characteristics; the place characteristics, environment characteristics and weather characteristics are processed by a feature processing network to obtain a third prediction value corresponding to the second historical data.

[0044] In some embodiments, the second training module 304 is further configured to calculate the total loss L according to the following formula, and optimize the weights in the student model according to the total loss: L=alpha * L1 + (1 - alpha) *L2; Among them, alpha is a preset hyperparameter, the range of alpha is between 0 and 1. The larger the aphha is, the more knowledge the student model uses the teacher model. L1 is the constraint loss, and L2 is the second prediction loss.

[0045] In some embodiments, the prediction module 305 is also configured to obtain the location data, environmental data and weather data of the second place at the current moment; input the location data, environmental data and weather data of the second place at the current moment into the student model, and inside the student model: process the location data, environmental data and weather data of the second place at the current moment respectively through the feature extraction network to obtain location characteristics, environmental characteristics and weather characteristics; process the location characteristics, environmental characteristics and weather characteristics through the feature processing network to obtain the cooling capacity required for the second place at the current moment.

[0046] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0047] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0048] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0049] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0050] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0051] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0052] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0053] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A refrigeration capacity prediction method based on knowledge distillation, characterized in that: include: A feature extraction network is constructed using a multi-head self-attention network, a feedforward neural network and a fully connected layer, a feature processing network is constructed using a fully connected layer and a Transformer module, and a teacher model and a student model are constructed using the feature extraction network and the feature processing network, wherein the teacher model and the student model have the same network structure, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; Acquire first historical data of a first location and second historical data of a second location, wherein the first historical data and the second historical data both include: location data, environment data, weather data, and cooling capacity; Using the first historical data to perform cooling capacity prediction training on the teacher model; Using the second historical data to perform the cooling capacity prediction training on the student model, and in the process of performing the cooling capacity prediction training on the student model, using the teacher model to constrain the student model; The student model is used to predict the cooling capacity required by the second location at different times of the day.

2. The method according to claim 1, characterized in that The feature extraction network is constructed using a multi-head self-attention network, a feedforward neural network, and a fully connected layer, including: Connecting the multi-head self-attention network, the first residual connection, the first batch normalization layer, the feedforward neural network, the first fully connected layer, the activation layer, the second fully connected layer, the second residual connection and the second batch normalization layer in sequence to obtain the feature extraction network; Among them, the first residual connection is used to connect the input of the first batch normalization layer with the input and output of the multi-head self-attention network, and the second residual connection is used to connect the input of the second batch normalization layer with the output of the second fully connected layer and the input of the multi-head self-attention network.

3. The method according to claim 1, characterized in that The feature processing network is constructed using the fully connected layer and Transformer module, including: The embedding layer, the fully connected layer, the batch normalization layer, the activation layer, the Transformer module, the fully connected layer, the batch normalization layer, the activation layer, the fully connected layer, the batch normalization layer and the activation layer are sequentially connected to obtain the feature processing network.

4. The method according to claim 1, characterized in that: Using the first historical data to perform cooling capacity prediction training on the teacher model includes: The location data, environment data and weather data in the first historical data are input into the teacher model, and within the teacher model: Processing the location data, environment data and weather data in the first historical data respectively through the feature extraction network to obtain location features, environment features and weather features; Processing the site feature, the environment feature, and the weather feature through the feature processing network to obtain a first prediction value corresponding to the first historical data; Calculating a first predicted loss between the first predicted value and the cooling capacity in the first historical data using a regression loss function; The weights in the teacher model are optimized based on the first prediction loss.

5. The method according to claim 1, characterized in that The method of performing the cooling capacity prediction training on the student model by using the second historical data and constraining the student model by using the teacher model during the cooling capacity prediction training on the student model includes: Inputting the location data, environment data and weather data in the second historical data into the teacher model, and outputting a second predicted value corresponding to the second historical data; Inputting the location data, environment data and weather data in the second historical data into the student model, and outputting a third predicted value corresponding to the second historical data; Calculating the constraint loss between the second predicted value and the third predicted value using a regression loss function; Calculating a second predicted loss between the third predicted value and the cooling capacity in the second historical data using the regression loss function; The weights in the student model are optimized according to the second prediction loss and the constraint loss.

6. The method according to claim 5, characterized in that Optimizing the weights in the student model according to the second prediction loss and the constraint loss includes: The total loss L is calculated according to the following formula, and the weights in the student model are optimized based on the total loss: L=alpha * L1 + (1 - alpha) *L2; Among them, alpha is a preset hyperparameter, the range of alpha is between 0 and 1, the larger the aphha is, the more knowledge the student model uses the teacher model, L1 is the constraint loss, and L2 is the second prediction loss.

7. The method according to claim 5, characterized in that Inputting the location data, environment data and weather data in the second historical data into the student model, and outputting a third predicted value corresponding to the second historical data, comprising: The location data, environment data and weather data in the second historical data are input into the student model, and within the student model: Processing the location data, environment data and weather data in the second historical data respectively through the feature extraction network to obtain location features, environment features and weather features; The location feature, the environment feature and the weather feature are processed by the feature processing network to obtain a third prediction value corresponding to the second historical data.

8. A refrigeration capacity prediction device based on knowledge distillation, characterized in that: include: A construction module is configured to construct a feature extraction network using a multi-head self-attention network, a feedforward neural network and a fully connected layer, construct a feature processing network using a fully connected layer and a Transformer module, and construct a teacher model and a student model using the feature extraction network and the feature processing network, wherein the teacher model and the student model have the same network structure, but the number of dimensions of the fully connected layer in the student model is less than the number of dimensions of the fully connected layer in the teacher model; An acquisition module is configured to acquire first historical data of a first location and second historical data of a second location, wherein the first historical data and the second historical data both include: location data, environment data, weather data and cooling capacity; A first training module is configured to perform cooling capacity prediction training on the teacher model using the first historical data; a second training module, configured to perform the refrigeration capacity prediction training on the student model using the second historical data, and in the process of performing the refrigeration capacity prediction training on the student model, constrain the student model using the teacher model; The prediction module is configured to use the student model to predict the cooling capacity required by the second place at different times of the day.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images