Training method, device, equipment and storage medium for spatiotemporal data processing model

Abstract

The present disclosure discloses a training method, device, equipment and storage medium for a spatiotemporal data processing model, and relates to the field of computer technology. The scheme is: extract the target spatiotemporal data of any area from a training data set, and the training data set includes the spatiotemporal data and attributes of each area; obtain the first reference spatiotemporal data of the first reference area with the same attributes as any area and the second reference spatiotemporal data of the second reference area with different attributes from any area; input the target, first and second reference spatiotemporal data into the model to be trained to obtain the first, second and third data representations; if the difference between the first and second data representations is less than the first threshold, and the difference between the first and second data representations is greater than the second threshold, the model training is terminated. Thus, by using the comparative self-supervised learning of spatiotemporal data to generate a spatiotemporal data processing model to process the spatiotemporal data, accurate and reliable data representations can be obtained, which can improve the accuracy of spatiotemporal prediction.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as deep learning and big data, and specifically to a training method, device, equipment and storage medium for a spatiotemporal data processing model. Background Art

[0002] With the development of mobile Internet, location-based services are widely present in our lives, generating a large amount of spatiotemporal data all the time, such as positioning data, mobile phone signaling data, wireless network data, etc. However, accurate and high-quality spatiotemporal data is usually small in quantity, difficult to obtain, and expensive.

[0003] Accurate spatiotemporal data repair is a very basic function in smart city applications. It can provide support for fine-grained passenger flow inference in scenic spots, offline smart marketing, smart traffic optimization and scheduling, crowd gathering warning and smart site selection. In related technologies, directly training models with a small amount of high-quality data can easily lead to overfitting of the model and fail to accurately reflect the real data situation. Therefore, how to improve the accuracy and reliability of spatiotemporal data is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a training method, device, equipment and storage medium for a spatiotemporal data processing model.

[0005] According to a first aspect of the present disclosure, a training method for a spatiotemporal data processing model is provided, comprising:

[0006] Extracting target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region;

[0007] Acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as the any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from the any area;

[0008] Inputting the target spatiotemporal data, the first reference spatiotemporal data, and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain a first data representation, a second data representation, and a third data representation, respectively;

[0009] When a first difference between the first data representation and the second data representation is smaller than a first threshold, and a second difference between the first data representation and the third data representation is larger than a second threshold, the training process of the spatiotemporal data processing model is terminated.

[0010] According to a second aspect of the present disclosure, a training device for a spatiotemporal data processing model is provided, comprising:

[0011] An extraction module, used to extract target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region;

[0012] A first acquisition module, configured to acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as the any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from the any area;

[0013] A second acquisition module, used to input the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain a first data representation, a second data representation and a third data representation, respectively;

[0014] A judgment module is used to end the training process of the spatiotemporal data processing model when a first difference between the first data representation and the second data representation is less than a first threshold and a second difference between the first data representation and the third data representation is greater than a second threshold.

[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the above-mentioned embodiment.

[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program is stored, wherein the computer instructions are used to cause the computer to execute the method described in the above-mentioned first embodiment.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0021] The training method, device, equipment and storage medium of the spatiotemporal data processing model provided by the present disclosure have at least the following beneficial effects:

[0022] In the disclosed embodiment, the device first extracts the target spatiotemporal data associated with any region from the training data set, wherein the training data set includes the spatiotemporal data associated with each region and the attribute information of each region, and then obtains the first reference spatiotemporal data associated with the first reference region with the same attribute information as any region, and the second reference spatiotemporal data associated with the second reference region with different attribute information from any region from the training data set, and then inputs the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain the first data representation, the second data representation and the third data representation, respectively. Finally, when the first difference between the first data representation and the second data representation is less than the first threshold value, and the second difference between the first data representation and the third data representation is greater than the second threshold value, the training process of the spatiotemporal data processing model is terminated. Thus, by utilizing the comparative self-supervised learning of spatiotemporal data, a spatiotemporal data processing model can be generated, so that the spatiotemporal data can be processed to obtain accurate and reliable spatiotemporal data representation, which provides conditions for improving the accuracy of spatiotemporal prediction when accurate spatiotemporal data is insufficient.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0025] Figure 1 It is a flowchart of a training method of a spatiotemporal data processing model provided by the present disclosure;

[0026] Figure 2 is a flow chart of another training method of a spatiotemporal data processing model provided by the present disclosure;

[0027] Figure 3 is a flow chart of another training method of a spatiotemporal data processing model provided by the present disclosure;

[0028] Figure 4 It is a framework diagram of model pre-training according to a training method of a spatiotemporal data processing model provided by the present disclosure;

[0029] Figure 5 A population inference framework diagram for multi-source data fusion provided by the present disclosure;

[0030] Figure 6 A structural block diagram of a training device for a spatiotemporal data processing model provided by the present disclosure;

[0031] Figure 7 A structural block diagram of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] The training method of the spatiotemporal data processing model proposed in the present disclosure can be executed by a training device based on the spatiotemporal data processing model provided by the present disclosure, and can also be executed by an electronic device provided by the present disclosure, wherein the electronic device can include but is not limited to terminal devices such as desktop computers and tablet computers, and can also be a server. Below, a training method of a spatiotemporal data processing model provided by the present disclosure is executed by a training device for a spatiotemporal data processing model provided by the present disclosure, and this is not a limitation of the present disclosure, hereinafter referred to as "device".

[0034] It should be noted that the user data disclosed in this disclosure are obtained in compliance with relevant laws and regulations.

[0035] The following is a detailed description of a training method, apparatus, computer equipment and storage medium for a spatiotemporal data processing model provided by the present disclosure with reference to the accompanying drawings.

[0036] Figure 1 It is a flowchart of a training method for a spatiotemporal data processing model according to an embodiment of the present disclosure.

[0037] like Figure 1 As shown, the training method of the spatiotemporal data processing model may include the following steps:

[0038] Step S101, extracting target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region.

[0039] It should be noted that spatiotemporal data has time characteristics and space characteristics. The time characteristics represent the characteristics of spatiotemporal data in the time dimension, and the space characteristics represent the characteristics of spatiotemporal data in the space characteristics. In the disclosed embodiment, the spatiotemporal data can be real-time positioning data, communication data, real-time population distribution data in a region, etc., which are not limited here.

[0040] The target spatiotemporal data is spatiotemporal data associated with any area, wherein any area may be a scenic spot, a school, a shopping mall or other land area, which is not limited here.

[0041] The attribute information of each area may include the location, area, flow of people, population data, traffic conditions, type, etc. of each area, which is not limited here.

[0042] Step S102: acquiring, from the training data set, first reference spatiotemporal data associated with a first reference region having the same attribute information as any region, and second reference spatiotemporal data associated with a second reference region having different attribute information from any region.

[0043] It should be noted that in order to better ensure the spatiotemporal data processing model, a small amount of data can be used for training to learn the differences between the spatiotemporal data corresponding to areas with different attribute information. Optionally, the device can obtain from the training data set first reference spatiotemporal data associated with a first reference area that has the same attribute information as any area, and obtain from the training data set second reference spatiotemporal data associated with a second reference area that has different attribute information from any area.

[0044] Alternatively, first reference spatiotemporal data associated with a first reference area having at least one attribute information identical to that of any area may be obtained from the training data set, and second reference spatiotemporal data associated with a second reference area having all attribute information different from that of any area may be obtained from the training data set. There is no limitation on this.

[0045] Optionally, the device may determine the first threshold value according to the first matching degree between the attribute information of any region and the attribute information of the first reference region. It is understandable that, since the attribute information of any region is the same as that of the first reference region, that is, any region should be the same or similar to the first reference region, the first matching degree of any region and the first reference region is relatively high, and the difference is small, so the first threshold value is small.

[0046] Optionally, the device may determine the second threshold value according to the second matching degree between the attribute information of any region and the attribute information of the second reference region. It is understandable that, since the attribute information of any region is different from that of the second reference region, that is, any region should be different from or have a large difference with the first reference region, the second matching degree of any region and the second reference region is relatively low, the difference is relatively large, and therefore the second threshold value is relatively large.

[0047] In step S103, the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

[0048] It is understandable that the first data representation can represent the characteristics of the target spatiotemporal data, which can be a vector, a matrix, etc., and the present disclosure does not limit this. Correspondingly, the second data representation can represent the first reference spatiotemporal data, and the third data representation can represent the second reference spatiotemporal data.

[0049] Step S104, when the first difference between the first data representation and the second data representation is less than the first threshold, and the second difference between the first data representation and the third data representation is greater than the second threshold, the training process of the spatiotemporal data processing model is terminated.

[0050] After obtaining the first data representation, the second data representation, and the third data representation, the device may calculate the difference between the first data representation and the second data representation, and between the first data representation and the third data representation. Specifically, the device may determine the first difference by comparing the first data representation and the second data representation, and determine the second difference by comparing the first data representation and the third data representation.

[0051] The first threshold is a threshold of a first difference degree determined according to the first matching degree, and the second threshold is a threshold of a second difference degree determined according to the second matching degree.

[0052] It can be understood that since the first reference area is an area with the same attribute information as any area, the data representations corresponding to the two areas will be relatively similar. Correspondingly, the second reference area is an area with different attribute information from any area, so the data representations corresponding to the two areas will be quite different. If the first difference is less than the first threshold, it means that the difference between the first reference area and any area is relatively small. If the second difference is greater than the second threshold, it means that the difference between any area and the second reference area is relatively large. Therefore, it can be explained that the data representation obtained using the current spatiotemporal data processing model is reasonable and reliable, which can meet the needs of the device to process the spatiotemporal area of ​​any area, so the training process of the spatiotemporal data processing model can be terminated.

[0053] In the disclosed embodiment, the device first extracts the target spatiotemporal data associated with any region from the training data set, wherein the training data set includes the spatiotemporal data associated with each region and the attribute information of each region, and then obtains the first reference spatiotemporal data associated with the first reference region with the same attribute information as any region, and the second reference spatiotemporal data associated with the second reference region with different attribute information from any region from the training data set, and then inputs the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain the first data representation, the second data representation and the third data representation, respectively. Finally, when the first difference between the first data representation and the second data representation is less than the first threshold value, and the second difference between the first data representation and the third data representation is greater than the second threshold value, the training process of the spatiotemporal data processing model is terminated. Thus, by utilizing the comparative self-supervised learning of spatiotemporal data, a spatiotemporal data processing model can be generated, so that the spatiotemporal data can be processed to obtain accurate and reliable spatiotemporal data representation, which provides conditions for improving the accuracy of spatiotemporal prediction when accurate spatiotemporal data is insufficient.

[0054] Figure 2 It is a flowchart of a training method for a spatiotemporal data processing model according to another embodiment of the present disclosure.

[0055] like Figure 2 As shown, the training method of the spatiotemporal data processing model may include the following steps:

[0056] Step S201: extracting target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region.

[0057] It should be noted that the specific implementation process of step S201 can refer to the detailed description of any embodiment of the present disclosure and will not be repeated here.

[0058] Step S202: acquiring, from the training data set, first reference spatiotemporal data associated with a first reference region having at least one attribute information identical to that of any region in the attribute information.

[0059] Optionally, at least one of the attribute information of the first reference area is the same as the attribute information between any areas. For example, the device can obtain the first reference space-time data associated with the first reference area that has the same area and traffic conditions as any area; or, the device can also obtain the first reference space-time data associated with the first reference area that has the same area, traffic conditions, and population gender distribution as any area, which is not limited here.

[0060] Step S203: acquiring, from the training data set, second reference spatiotemporal data associated with a second reference region having different attribute information from the attribute information of any region.

[0061] It should be noted that in order to better ensure the spatiotemporal data processing model, a small amount of data can be used for training to learn the differences between the spatiotemporal data corresponding to areas with different attribute information. In the present disclosure, when the first reference area has at least one attribute information that is the same as the attribute information of any area, the device can use an area that is different from all the attribute information in the attribute information of any area as the second reference area.

[0062] Optionally, in the present disclosure, first reference spatiotemporal data associated with a first reference area having the same attribute information as any area can be obtained from the training data set; correspondingly, second reference spatiotemporal data associated with a second reference area having at least one attribute information different from the attribute information of any area can be obtained from the training data set.

[0063] In step S204, the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

[0064] It should be noted that the specific implementation process of step S204 can refer to the above embodiment and will not be described in detail here.

[0065] Step S205, when the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, or the second difference between the first data representation and the third data representation is less than or equal to the second threshold, return to execute the operation of extracting the target spatiotemporal data from the training data set, until the first difference between the first data representation newly generated by the spatiotemporal data processing model to be trained and the newly generated second data representation is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, then the training process of the spatiotemporal data processing model is ended.

[0066] It should be noted that, if the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, it means that the difference between the spatiotemporal data representation of any area obtained after processing by the spatiotemporal data processing model and the first reference area is large. Since the attribute information of any area is the same as that of the first reference area, that is, the spatiotemporal data representations corresponding to the two should be the same or similar, it can be explained that the spatiotemporal data processing model at this time is inaccurate and cannot meet the conditions for ending the training of the neural network training model. The device will continue to return to execute the operation of extracting the target spatiotemporal data from the training data set.

[0067] Alternatively, if the second difference between the first data representation and the third data representation is less than or equal to the second threshold, it means that the difference between the spatiotemporal data representation of any area obtained after processing by the spatiotemporal data processing model and the second reference area is small. However, since the attribute information of any area is different from that of the second reference area, the spatiotemporal data representations corresponding to the two should be different or have a large difference, which means that the spatiotemporal data processing model at this time is inaccurate and cannot meet the conditions for ending the training of the neural network training model. The device will then continue to return to execute the operation of extracting the target spatiotemporal data from the training data set.

[0068] Furthermore, after returning to execute the operation of extracting the target spatiotemporal data from the training data set, if the first difference between the newly generated first data representation and the newly generated second data representation of the spatiotemporal data processing model to be trained is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, the training process of the spatiotemporal data processing model can be ended.

[0069] In the disclosed embodiment, the device first extracts target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region, and then obtains first reference spatiotemporal data associated with a first reference region having at least one attribute information identical to that in the attribute information of any region from the training data set, and obtains second reference spatiotemporal data associated with a second reference region having all attribute information different from that in the attribute information of any region from the training data set, and then inputs the target spatiotemporal data, the first reference spatiotemporal data, and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to respectively Obtain the first data representation, the second data representation and the third data representation, and finally, when the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, or the second difference between the first data representation and the third data representation is less than or equal to the second threshold, return to execute the operation of extracting the target spatiotemporal data from the training data set, until the first difference between the first data representation newly generated by the spatiotemporal data processing model to be trained and the newly generated second data representation is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, the training process of the spatiotemporal data processing model is ended. Thus, by using contrastive self-supervised learning, a large amount of low-quality data can be used to assist in the training of the spatiotemporal data processing model, so that the low-quality spatiotemporal data can be processed by using the moment data processing model generated by training to obtain a more accurate and reliable spatiotemporal data representation.

[0070] Figure 3 It is a flowchart of a training method for a spatiotemporal data processing model according to another embodiment of the present disclosure.

[0071] like Figure 3 As shown, the training method of the spatiotemporal data processing model may include the following steps:

[0072] Step S301: extracting target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region.

[0073] Optionally, the spatiotemporal data associated with each region may include a communication data sequence, an attribute map, a population attribute, and a positioning data sequence corresponding to each region, which is not limited to this.

[0074] It should be noted that the attribute map can be constructed by the device based on the attribute information of the known area. For example, the area can be used as a node, the road network distance can be used as the edge length, and the attribute information of the area, such as housing prices and passenger flow, can be used as the attribute of each node. There is no limitation on this. The population attribute can be obtained by the device based on the analysis of the acquired positioning data set communication data, for example, it can be the population age distribution, population gender distribution, etc. There is no limitation on this.

[0075] Optionally, the training data set may also include annotation labels corresponding to each area, such as the traffic conditions of each area, whether a gathering event occurs, etc., which is not limited.

[0076] Step S302: acquiring, from the training data set, first reference spatiotemporal data associated with a first reference region having the same attribute information as any region, and second reference spatiotemporal data associated with a second reference region having different attribute information from any region.

[0077] It should be noted that the specific implementation process of step S302 may refer to any of the above embodiments and will not be described in detail here.

[0078] Step S303, using the knowledge fusion network to be trained, fuses the communication data sequences, attribute maps, population attributes and positioning data sequences corresponding to any area, the first reference area and the second reference area respectively to obtain the fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data.

[0079] In the present disclosure, the knowledge fusion network to be trained can be used to fuse the communication data sequences, attribute maps, population attributes, positioning data sequences and other data corresponding to any area, the first reference area and the second reference area respectively using the knowledge fusion network to be trained, so that the fused target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data can have the spatiotemporal data and attribute information of the corresponding areas.

[0080] Optionally, in the present disclosure, a Swapped Contrastive Loss mechanism can be used to predict the spatiotemporal data representation of one region from the spatiotemporal data representation of another region, so that a smaller batch of data can be used to train the network, reducing the memory requirement during model training.

[0081] like Figure 4 As shown, the present disclosure can adopt Figure 4 The framework diagram of the model pre-training shown in Figure 1 uses the exchange contrast loss mechanism to train the model. Figure 4 As shown in the figure, the training framework of the spatiotemporal data processing model can be divided into two types of networks. One is the prototype network, which is a full-connection network that mainly undertakes the task of representation calculation and shares network parameters when calculating target data and enhanced data. The other is the backbone network, which mainly completes the tasks of multi-source data fusion and feature extraction.

[0082] It should be noted that the backbone network Figure 4 Corresponding to the spatiotemporal knowledge fusion network in Figure 4 Corresponding to the encoding network in .

[0083] Specifically, swapped contrastive loss can be used to optimize the parameters of the network, and its expression can be:

[0084] L(o t ,o s )=L(o t ,q s )+L(q t ,o s )

[0085] Among them, t and s Can be the output of the backbone network, q t and q s It can be the output of the prototype network (prototype), and L is the cross entropy loss function.

[0086] It should be noted that L(o t ,q s ) can be calculated by the following formula:

[0087]

[0088]

[0089] Among them, τ is the temperature parameter and k is the number of prototype networks.

[0090] For the sake of convenience, let i be the node corresponding to any region and edge d be i , represents the distance between the adjacent node j and any region, with P a and P c As two intrinsic features of the nodes corresponding to each region, each P a It includes the area, traffic conditions and location of the area. c Contains regional population profiles, such as age and gender, and dynamic mobile phone positioning reporting data The flow of people in the reaction area is not limited.

[0091] Furthermore, for P a and P c Encoding, the specific formula is as follows:

[0092]

[0093]

[0094] in, represents the input of the i-th node in the network, and Represents two embedding networks f a and f c The input, θ a and θ c They represent the parameters of the two embedding networks respectively.

[0095] After that, the message passing neural network (MPNN) can be used for representation learning, where the edge network can be expressed as:

[0096]

[0097] Among them, w ij is the input of the edge network, θ e are the parameters to be learned, and finally the message passing network can be defined as:

[0098]

[0099] Among them, θ g Parameters for the readout network.

[0100] By utilizing the exchange contrast loss mechanism, the network parameters of the knowledge fusion network to be trained can be optimized, thereby obtaining more accurate target spatiotemporal data, first reference spatiotemporal data and second reference spatiotemporal data.

[0101] It should be noted that the above examples are only for illustrative purposes and the present disclosure does not limit them.

[0102] In step S304, the fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

[0103] Step S305, when the first difference between the first data representation and the second data representation is less than the first threshold, and the second difference between the first data representation and the third data representation is greater than the second threshold, the training process of the spatiotemporal data processing model is terminated.

[0104] It should be noted that the specific implementation process of S304 and S305 can refer to any of the above embodiments and will not be described in detail here.

[0105] Step S306, inputting the spatiotemporal data associated with each region into the spatiotemporal data processing model generated by training to obtain the data representation corresponding to each region.

[0106] It should be noted that after the training process of the spatiotemporal data processing model is completed, the device can input the spatiotemporal data associated with each region into the spatiotemporal data processing model generated by the training, so as to obtain the data representation corresponding to each region. It is understandable that the data representation can represent the spatiotemporal data associated with each region in the form of a static graph structure or a vector, and this is not limited.

[0107] As an example, Figure 5 A population inference framework diagram of multi-source data fusion is shown. In the above embodiment, before obtaining the spatiotemporal data and attribute information of each region, it is necessary to extract and construct the spatiotemporal features of various types of data in each region to obtain the spatiotemporal data such as communication data sequence, attribute spectrum, population attribute and positioning data sequence corresponding to each region. Figure 5As shown, various types of data can be e-commerce data, travel data, rental data, road network data, and Point of Interest (POI) data, etc. The device can extract and construct spatiotemporal features of various types of data information in various regions to obtain spatiotemporal data such as attribute maps, communication data sequences, and positioning data sequences. After that, by comparing the self-supervisory model training, the spatiotemporal deep knowledge fusion network, that is, the knowledge fusion network in the above embodiment, is used to fuse the communication data sequence, attribute map, population attribute, and positioning data sequence corresponding to any area, the first reference area, and the second reference area, respectively, and then the above-mentioned spatiotemporal data processing model is used to take each fused data as input to obtain each data representation. Finally, the device can input each data representation into the spatiotemporal predictor to obtain the predicted flow of people in each area, that is, the inferred value.

[0108] Step S307 , inputting the data representation corresponding to each region into the prediction model to be trained to obtain the prediction label corresponding to each region.

[0109] It should be noted that the prediction label can be a prediction label for the attribute information of each area, such as pedestrian flow prediction, traffic condition prediction, gender distribution prediction, etc., or it can also be information such as crowd gathering, marketing decisions, traffic scheduling decisions, etc., without limitation.

[0110] It is understandable that for different specific tasks, the corresponding prediction labels obtained by the device through the prediction model may also be different. Specifically, for the task of predicting human flow, the obtained prediction label may be the human flow corresponding to the area, and there is no limitation on this.

[0111] Step S308, according to the difference between the predicted label and the labeled label, the prediction model and the spatiotemporal data processing model generated by training are respectively corrected.

[0112] Specifically, the device can compare the predicted label with the annotated label to determine the difference between the predicted label and the annotated label. For example, the correction gradient can be determined by using gradient descent, stochastic gradient descent, etc., and then the prediction model and the spatiotemporal data processing model generated by training are corrected respectively. The present disclosure does not limit the method for determining the correction gradient.

[0113] Optionally, the parameters of the spatiotemporal data processing model can be modified using a regression network through the cross loss of the knowledge fusion network.

[0114] Optionally, you can use a previously trained backbone network and another regression network to be trained to perform fine-tuning and inference on specific tasks. The specific formula is as follows:

[0115]

[0116] Among them, t is the output of the backbone network, θ o is the regression network f o , σ is the parameter of the sigmoid activation function. In order to better train the model, a full training strategy can be used to continue training the backbone network and the regression network together under labeled data, which can be formally expressed as:

[0117]

[0118] θ b ={θ a ,θ c ,θ e ,θ g ,θ n ,θ s}

[0119] Among them, L is the loss function, α is the learning rate, and η is the decay coefficient.

[0120] In the disclosed embodiment, the device first extracts target spatiotemporal data associated with any area from a training data set, wherein the training data set includes spatiotemporal data associated with each area and attribute information of each area, and then obtains first reference spatiotemporal data associated with a first reference area having the same attribute information as any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from any area from the training data set, and then uses the knowledge fusion network to be trained to fuse the communication data sequence, attribute map, population attribute and positioning data sequence corresponding to any area, the first reference area and the second reference area, respectively, to obtain the fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data, and when the first difference between the first data representation and the second data representation is less than the first threshold value, and the second difference between the first data representation and the third data representation is greater than the second threshold value, the training process of the spatiotemporal data processing model is ended, and then the spatiotemporal data associated with each area is input into the spatiotemporal data processing model generated by the training to obtain the data representation corresponding to each area, and finally the data representation corresponding to each area is input into the prediction model to be trained to obtain the prediction label corresponding to each area. Therefore, through the fusion of multi-source spatiotemporal data, the accuracy of the prediction model can be improved, and the model can be fine-tuned while optimizing the model, so that the purpose of training with a small amount of data can be achieved to approximate the effect of the full amount of training data.

[0121] In order to implement the above embodiment, the embodiment of the present disclosure also proposes a training device for a spatiotemporal data processing model. Figure 6A structural block diagram of a training device for a spatiotemporal data processing model provided in an embodiment of the present disclosure.

[0122] like Figure 6 As shown, the training device of the spatiotemporal data processing model includes: an extraction module 610, a first acquisition module 620, a second acquisition module 630 and a judgment module 640.

[0123] An extraction module, used to extract target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region;

[0124] A first acquisition module, configured to acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as the any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from the any area;

[0125] A second acquisition module, used to input the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain a first data representation, a second data representation and a third data representation, respectively;

[0126] A judgment module is used to end the training process of the spatiotemporal data processing model when a first difference between the first data representation and the second data representation is less than a first threshold and a second difference between the first data representation and the third data representation is greater than a second threshold.

[0127] Optionally, the attribute information of the region includes the area, population and type of the region, and the first acquisition module is specifically used to:

[0128] Acquire, from the training data set, first reference spatiotemporal data associated with a first reference region having the same at least one attribute information as that of the attribute information of any one region;

[0129] Second reference spatiotemporal data associated with a second reference area having different attribute information from the attribute information of any area is obtained from the training data set.

[0130] Optionally, the attribute information of the region includes the area, population and type of the region, and the first acquisition module is specifically used to:

[0131] Acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as any one of the areas;

[0132] Second reference spatiotemporal data associated with a second reference area having at least one attribute information different from the attribute information of any one of the areas is obtained from the training data set.

[0133] Optionally, the device further includes:

[0134] A first determination module, configured to determine the first threshold value according to a first matching degree between the attribute information of any one area and the attribute information of the first reference area;

[0135] The second determination module is used to determine the second threshold according to a second matching degree between the attribute information of any one area and the attribute information of the second reference area.

[0136] Optionally, the judging module is further used to:

[0137] When the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, or the second difference between the first data representation and the third data representation is less than or equal to the second threshold, return to executing the operation of extracting target spatiotemporal data from the training data set until the first difference between the first data representation newly generated by the spatiotemporal data processing model to be trained and the newly generated second data representation is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, then the training process of the spatiotemporal data processing model is terminated.

[0138] Optionally, the training data set also includes a label corresponding to each of the regions, and the device further includes:

[0139] A third acquisition module is used to input the spatiotemporal data associated with each of the regions into the spatiotemporal data processing model generated by training to obtain data representation corresponding to each of the regions;

[0140] A fourth acquisition module, used for inputting the data representation corresponding to each of the regions into the prediction model to be trained, so as to obtain the prediction label corresponding to each of the regions;

[0141] The correction module is used to correct the prediction model and the spatiotemporal data processing model generated by the training according to the difference between the predicted label and the labeled label.

[0142] Optionally, the spatiotemporal data associated with each of the regions includes a communication data sequence, an attribute map, a population attribute and a positioning data sequence corresponding to each region, and the second acquisition module is further used to:

[0143] Using the knowledge fusion network to be trained, the communication data sequence, attribute map, population attribute and positioning data sequence corresponding to any one of the regions, the first reference region and the second reference region are fused to obtain fused target spatiotemporal data, fused first reference spatiotemporal data and second reference spatiotemporal data;

[0144] The fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

[0145] In the disclosed embodiment, the device first extracts the target spatiotemporal data associated with any region from the training data set, wherein the training data set includes the spatiotemporal data associated with each region and the attribute information of each region, and then obtains the first reference spatiotemporal data associated with the first reference region with the same attribute information as any region, and the second reference spatiotemporal data associated with the second reference region with different attribute information from any region from the training data set, and then inputs the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain the first data representation, the second data representation and the third data representation, respectively. Finally, when the first difference between the first data representation and the second data representation is less than the first threshold value, and the second difference between the first data representation and the third data representation is greater than the second threshold value, the training process of the spatiotemporal data processing model is terminated. Thus, by utilizing the comparative self-supervised learning of spatiotemporal data, a spatiotemporal data processing model can be generated, so that the spatiotemporal data can be processed to obtain accurate and reliable spatiotemporal data representation, which provides conditions for improving the accuracy of spatiotemporal prediction when accurate spatiotemporal data is insufficient.

[0146] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0147] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0148] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0149] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the training method of the spatiotemporal data processing model. For example, in some embodiments, the training method of the spatiotemporal data processing model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the training method of the spatiotemporal data processing model described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the training method of the spatiotemporal data processing model in any other appropriate manner (eg, by means of firmware).

[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0156] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0157] In the disclosed embodiment, the device first extracts the target spatiotemporal data associated with any region from the training data set, wherein the training data set includes the spatiotemporal data associated with each region and the attribute information of each region, and then obtains the first reference spatiotemporal data associated with the first reference region with the same attribute information as any region, and the second reference spatiotemporal data associated with the second reference region with different attribute information from any region from the training data set, and then inputs the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain the first data representation, the second data representation and the third data representation, respectively. Finally, when the first difference between the first data representation and the second data representation is less than the first threshold value, and the second difference between the first data representation and the third data representation is greater than the second threshold value, the training process of the spatiotemporal data processing model is terminated. Thus, by utilizing the comparative self-supervised learning of spatiotemporal data, a spatiotemporal data processing model can be generated, so that the spatiotemporal data can be processed to obtain accurate and reliable spatiotemporal data representation, which provides conditions for improving the accuracy of spatiotemporal prediction when accurate spatiotemporal data is insufficient.

[0158] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0159] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a spatiotemporal data processing model, characterized in that: include: Extracting target spatiotemporal data associated with any region from a training data set, wherein the training data set includes spatiotemporal data associated with each region and attribute information of each region, and the spatiotemporal data associated with each region includes a communication data sequence, an attribute map, a population attribute, and a positioning data sequence corresponding to each region; Acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as the any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from the any area; Inputting the target spatiotemporal data, the first reference spatiotemporal data, and the second reference spatiotemporal data into a spatiotemporal data processing model to be trained, respectively, to obtain a first data representation, a second data representation, and a third data representation, respectively, wherein the first data representation is used to represent the characteristics of the target spatiotemporal data, the second data representation is used to represent the first reference spatiotemporal data, and the third data representation is used to represent the second reference spatiotemporal data; When a first difference between the first data representation and the second data representation is smaller than a first threshold, and a second difference between the first data representation and the third data representation is larger than a second threshold, the training process of the spatiotemporal data processing model is terminated.

2. The method according to claim 1, characterized in that The attribute information of the region includes the area, population and type of the region, and the obtaining, from the training data set, first reference spatiotemporal data associated with a first reference region having the same attribute information as any of the regions, and second reference spatiotemporal data associated with a second reference region having different attribute information from any of the regions, includes: Acquire, from the training data set, first reference spatiotemporal data associated with a first reference region having the same at least one attribute information as that of the attribute information of any one region; Second reference spatiotemporal data associated with a second reference area having different attribute information from the attribute information of any area is obtained from the training data set.

3. The method according to claim 1, characterized in that The attribute information of the region includes the area, population and type of the region, and the obtaining, from the training data set, first reference spatiotemporal data associated with a first reference region having the same attribute information as any of the regions, and second reference spatiotemporal data associated with a second reference region having different attribute information from any of the regions, includes: Acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as any one of the areas; Second reference spatiotemporal data associated with a second reference area having at least one attribute information different from the attribute information of any one of the areas is obtained from the training data set.

4. The method according to claim 3, characterized in that Also includes: determining the first threshold value according to a first matching degree between the attribute information of any one area and the attribute information of the first reference area; The first threshold is determined according to a second matching degree between the attribute information of any one area and the attribute information of the second reference area.

5. The method according to claim 1, characterized in that After obtaining the first data representation, the second data representation and the third data representation, the method further includes: When the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, or the second difference between the first data representation and the third data representation is less than or equal to the second threshold, return to executing the operation of extracting target spatiotemporal data from the training data set until the first difference between the first data representation newly generated by the spatiotemporal data processing model to be trained and the newly generated second data representation is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, then the training process of the spatiotemporal data processing model is terminated.

6. The method according to any one of claims 1 to 5, characterized in that: The training data set also includes a label corresponding to each of the regions, and the method further includes: Inputting the spatiotemporal data associated with each of the regions into the spatiotemporal data processing model generated by training to obtain data representation corresponding to each of the regions; Inputting the data representation corresponding to each of the regions into the prediction model to be trained to obtain a prediction label corresponding to each of the regions; According to the difference between the predicted label and the labeled label, the prediction model and the spatiotemporal data processing model generated by the training are respectively modified.

7. The method according to claim 6, characterized in that The step of inputting the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained comprises: Using the knowledge fusion network to be trained, the communication data sequence, attribute map, population attribute and positioning data sequence corresponding to any one of the regions, the first reference region and the second reference region are fused to obtain fused target spatiotemporal data, fused first reference spatiotemporal data and second reference spatiotemporal data; The fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

8. A training device for a spatiotemporal data processing model, characterized in that: include: An extraction module is used to extract target spatiotemporal data associated with any area from a training data set, wherein the training data set includes spatiotemporal data associated with each area and attribute information of each area, and the spatiotemporal data associated with each area includes a communication data sequence, an attribute map, a population attribute, and a positioning data sequence corresponding to each area; A first acquisition module, configured to acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as the any area, and second reference spatiotemporal data associated with a second reference area having different attribute information from the any area; A second acquisition module, used to input the target spatiotemporal data, the first reference spatiotemporal data and the second reference spatiotemporal data into the spatiotemporal data processing model to be trained, respectively, to obtain a first data representation, a second data representation and a third data representation, respectively, wherein the first data representation is used to represent the characteristics of the target spatiotemporal data, the second data representation is used to represent the first reference spatiotemporal data, and the third data representation is used to represent the second reference spatiotemporal data; A judgment module is used to end the training process of the spatiotemporal data processing model when a first difference between the first data representation and the second data representation is less than a first threshold and a second difference between the first data representation and the third data representation is greater than a second threshold.

9. The device according to claim 8, characterized in that The attribute information of the region includes the area, population and type of the region. The first acquisition module is specifically used to: Acquire, from the training data set, first reference spatiotemporal data associated with a first reference region having the same at least one attribute information as that of the attribute information of any one region; Second reference spatiotemporal data associated with a second reference area having different attribute information from the attribute information of any area is obtained from the training data set.

10. The device according to claim 8, characterized in that The attribute information of the region includes the area, population and type of the region. The first acquisition module is specifically used to: Acquire, from the training data set, first reference spatiotemporal data associated with a first reference area having the same attribute information as any one of the areas; Second reference spatiotemporal data associated with a second reference area having at least one attribute information different from the attribute information of any one of the areas is obtained from the training data set.

11. The device according to claim 10, characterized in that Also includes: A first determination module, configured to determine the first threshold value according to a first matching degree between the attribute information of any one area and the attribute information of the first reference area; The second determination module is used to determine the second threshold according to a second matching degree between the attribute information of any one area and the attribute information of the second reference area.

12. The device according to claim 8, characterized in that The judging module is further used for: When the first difference between the first data representation and the second data representation is greater than or equal to the first threshold, or the second difference between the first data representation and the third data representation is less than or equal to the second threshold, return to executing the operation of extracting target spatiotemporal data from the training data set until the first difference between the first data representation newly generated by the spatiotemporal data processing model to be trained and the newly generated second data representation is less than the first threshold, and the second difference between the first data representation and the newly generated third data representation is greater than the second threshold, then the training process of the spatiotemporal data processing model is terminated.

13. The device according to any one of claims 8 to 12, characterized in that: The training data set also includes a label corresponding to each of the regions, and the device also includes: A third acquisition module is used to input the spatiotemporal data associated with each of the regions into the spatiotemporal data processing model generated by training to obtain data representation corresponding to each of the regions; A fourth acquisition module, used for inputting the data representation corresponding to each of the regions into the prediction model to be trained, so as to obtain the prediction label corresponding to each of the regions; The correction module is used to correct the prediction model and the spatiotemporal data processing model generated by the training according to the difference between the predicted label and the labeled label.

14. The device according to claim 13, characterized in that The second acquisition module is further used for: Using the knowledge fusion network to be trained, the communication data sequence, attribute map, population attribute and positioning data sequence corresponding to any one of the regions, the first reference region and the second reference region are fused to obtain fused target spatiotemporal data, fused first reference spatiotemporal data and second reference spatiotemporal data; The fused target spatiotemporal data, the fused first reference spatiotemporal data and the second reference spatiotemporal data are respectively input into the spatiotemporal data processing model to be trained to obtain the first data representation, the second data representation and the third data representation respectively.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images