Information processing method and device, cloud equipment and storage medium

Abstract

The invention discloses an information processing method and device, cloud equipment and a storage medium. The method comprises the following steps: constructing a first model; training the first model by using the acquired first image data, the second image data, the second model, the third model and the fourth model of the at least two target weather domains; the trained first model is sent to a monitoring system, first data are matched with second data for the to-be-recognized image data of the target weather domain, the first data comprise first semantic features and corresponding weather domain labels, and the second data comprise second semantic features and corresponding weather domain labels. According to the scheme provided by the embodiment of the invention, the model suitable for multiple weather conditions can be obtained only through one-time migration, so that the time required for training is shortened, the computing power cost is reduced, and the storage space is saved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to an information processing method, apparatus, cloud device and storage medium. Background Art

[0002] Video surveillance is an important part of the security system and the physical basis for real-time monitoring activities in important departments or places. In the video surveillance system, deep learning methods based on neural networks are used to analyze videos to identify objects in video images, classify objects, and determine the attributes of objects to assist security work, thus greatly reducing labor costs.

[0003] Weather changes are an important cause of backlighting, low illumination, and interference from irrelevant objects in surveillance videos, which seriously affects the quality and analysis of videos. In order to cope with complex and diverse weather characteristics, it is often necessary to pre-process video images in a targeted manner and use a large amount of additional data and labels to train the neural network in the video surveillance system, which greatly increases the cost of system operation. Nowadays, video processing algorithms mainly use transfer learning to solve the problem of lack of data labels. Transfer learning trains on the basis of the source network by exploring and utilizing the correlation between similar data, so that the newly obtained network parameters can be applied to other fields with similar tasks to the source network. For example, the semantic feature extraction network suitable for sunny scenes can be applied to the semantic feature extraction tasks of foggy scenes and even thunderstorm scenes after transfer learning.

[0004] In the prior art, when there is more than one target domain for transfer learning, for example, to obtain a neural network suitable for multiple weather conditions, it is necessary to migrate from a single weather domain to another single weather domain multiple times to obtain neural network parameters suitable for multiple weather conditions. Each migration will require a lot of computing power and time costs. At the same time, for semantic feature extraction tasks in different weather conditions, the neural network needs to load different network parameters. Storing multiple network parameters requires additional storage space, and sometimes it is necessary to train additional networks to realize network parameter loading under different weather conditions, resulting in additional overhead. Summary of the invention

[0005] To solve related technical problems, the embodiments of the present application provide an information processing method, apparatus, related equipment and storage medium.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present application provides an information processing method, which is applied to the cloud, including:

[0008] Build the first model;

[0009] The first model is trained using the acquired first image data, second image data, second model, third model, and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model;

[0010] The trained first model is sent to the monitoring system, wherein, for image data to be identified in the target weather domain, the first data is matched with the second data, the first data comprises a first semantic feature and a corresponding weather domain label, the first semantic feature comprises a semantic feature extracted by the trained first model, and the second data comprises a second semantic feature and a corresponding weather domain label, the second semantic feature comprises a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

[0011] In the above solution, the method of training the first model using the first image data, the second image data, the second model, the third model, and the fourth model of at least two target weather domains obtained includes:

[0012] For each image data in the first image data, extracting semantic features of the image data using the first model;

[0013] Predicting the weather domain to which the image data belongs by using the third model and the semantic features of the image data;

[0014] The weather domain to which the image data corresponding to the semantic feature belongs is determined by using the semantic features of the image data and the corresponding weather domain, as well as the third data and the fourth model, wherein the third data includes the semantic features of the image data and the source weather domain obtained by using the second model and the second image data; wherein,

[0015] The parameters of the first model, the third model and the fourth model are optimized by the loss function.

[0016] In the above scheme, the parameters of the first model and the third model are optimized simultaneously by the first loss function, so that the predicted weather domain is the same as the weather domain to which the image data actually belongs;

[0017] Optimizing the parameters of the fourth model by using the second loss function so that the weather domain predicted by the fourth model is the same as the weather domain to which the image data actually belongs;

[0018] The parameters of the first model are optimized by a third loss function so that the first data matches the second data.

[0019] In the above scheme, the method further comprises:

[0020] For each image data in the first image data, the predicted weather domain is used as the weather domain label of the image data; wherein,

[0021] For each image data in the second image data, the source weather domain is a weather domain label of the image data.

[0022] In the above solution, for each image data in the second image data, the weather domain label of the image data is preset.

[0023] In the above solution, the constructing of the first model includes:

[0024] The first model is constructed using the second model.

[0025] In the above solution, using the second model to construct the first model includes:

[0026] Determine the structure of the first model using the structure of the second model;

[0027] The parameters of the second model are determined as the parameters of the first model.

[0028] The present application also provides an information processing device, including:

[0029] A construction module, used for constructing a first model;

[0030] a training module, used for training the first model by using the acquired first image data, second image data, second model, third model and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used for extracting semantic features of the second image data, the third model is used for predicting the weather domain to which the first image data belongs, the fourth model is used for judging whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model;

[0031] A sending module is used to send the trained first model to a monitoring system, wherein, for image data to be identified in a target weather domain, the first data is matched with the second data, the first data includes a first semantic feature and a corresponding weather domain label, the first semantic feature includes a semantic feature extracted by the trained first model, and the second data includes a second semantic feature and a corresponding weather domain label, the second semantic feature includes a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

[0032] The embodiment of the present application further provides a cloud device, comprising: a processor and a memory for storing a computer program that can be run on the processor,

[0033] Wherein, the processor is used to execute the steps of any of the above methods when running the computer program.

[0034] An embodiment of the present application further provides a storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0035] The information processing method, apparatus, cloud device and storage medium provided in the embodiments of the present application construct a first model in the cloud; train the first model using the acquired first image data, second image data, second model, third model and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, and the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, the input data includes the semantic features output by the second model and the semantic features output by the first model; send the trained first model to the monitoring system, wherein, for the image data to be identified in the target weather domain, the first data matches the second data, the first data includes the first semantic features and the corresponding weather domain label, the first semantic features include the semantic features extracted by the trained first model, the second data includes the second semantic features and the corresponding weather domain label, and the second semantic features include the semantic features extracted by the second model when the weather domain of the image data to be identified is replaced with the source weather domain. The solution provided by the embodiment of the present application is to build a semantic feature extraction model applicable to multiple target weather domains in the cloud; use image data of a source weather domain, image data of at least two target weather domains including the source weather domain, a model for extracting semantic features of monitoring data of the source weather domain, a model for predicting the weather domain to which the image data belongs, and a model that can determine whether the input feature data comes from the source weather domain or the target weather domain to jointly train a semantic feature extraction model applicable to monitoring data of multiple target weather domains; and send the semantic feature extraction model applicable to monitoring data of multiple target weather domains trained in the cloud to the monitoring system for deployment. In this way, only one migration is needed to obtain a model applicable to multiple weather conditions, thereby realizing simultaneous migration to multiple target weather domains, reducing the time required for training, and reducing computing power costs; at the same time, one model can be applicable to multiple weather conditions, and only the parameters of one model need to be stored, thus saving storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of an information processing method according to an embodiment of the present application;

[0037] Figure 2 This is a schematic diagram of the structure of an outdoor video surveillance system according to an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the process of training a source weather domain network model in an embodiment of the present application;

[0039] Figure 4A schematic diagram of the process of training the first model and the third model in the embodiment of the present application;

[0040] Figure 5 This is a schematic diagram of the structure of the fourth model of the embodiment of the present application;

[0041] Figure 6 A schematic diagram of the process of training the first model and the fourth model in the embodiment of the present application;

[0042] Figure 7 This is a schematic diagram of the structure of a feature-level discriminator according to an embodiment of the present application;

[0043] Figure 8 This is a schematic diagram of an intelligent monitoring and management system for coping with changeable weather based on multi-domain transfer learning as an example of the application of this application;

[0044] Fig. 9 A flowchart of a semi-supervised domain adaptive information processing method for simultaneous migration from a single weather domain to a multi-target weather domain based on a conditional generative adversarial network (CGAN) adversarial loss is provided as an example of the application of this application;

[0045] Fig.10 This is a schematic diagram of the structure of the source weather domain network for this application example;

[0046] Fig.11 This is a schematic diagram of the CGAN structure used for multi-target weather domain migration in this application example;

[0047] Fig.12 A schematic diagram of the structure of the multi-weather semantic segmentation network built for the application example of this application;

[0048] Fig.13 This is a schematic diagram of the structure of the intelligent monitoring management platform used in this application example;

[0049] Fig.14 This is a schematic diagram of the structure of an information processing device according to an embodiment of the present application;

[0050] Fig.15 This is a schematic diagram of the cloud device structure of an embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application will be further described in detail below through the accompanying drawings and specific embodiments.

[0052] Video surveillance is an important part of the security system and is the physical basis for real-time monitoring activities in important departments or places. In the video surveillance system, deep learning methods based on neural networks are used to analyze videos, which can include: identifying objects in video images, classifying identified objects, determining the attributes of identified objects, etc. The analysis results can effectively assist security work, thus greatly reducing labor costs.

[0053] However, training neural networks requires large data sets to identify similar and dissimilar attributes between data and determine how to weigh the correlations between said attributes. The training process requires a lot of computing power, far more than the computing power required for local surveillance systems to detect and classify objects in videos in field use.

[0054] At the same time, the changing external environmental factors and the interference of the imaging system will seriously affect the accuracy of semantic feature extraction of the neural network. For video surveillance systems located outdoors, weather changes are an important reason for the appearance of backlight, low illumination, and interference from irrelevant objects in the surveillance video, which seriously affects the quality of the video and makes it difficult to analyze the video. For example, cloudy days will cause the contrast of the face in the video image to decrease, and the edge contour information will be blurred, which will cause serious interference in the recognition of the face; rain will cause the vehicle imaging in the video image to be blurred and the information to be covered; foggy days will cause the color of the municipal image to be dim, and some important safety hazards will be difficult to distinguish. In order to cope with the complex and diverse weather characteristics, it is often necessary to pre-process the video images in a targeted manner and use a large amount of additional data and labels to train the neural network in the video surveillance system, which greatly increases the cost of system operation.

[0055] In the related art, transfer learning is mainly used to solve the problem of lack of labels for video data used to train neural networks. Transfer learning explores and utilizes the correlation between similar data, and trains based on the network applicable to the source domain, so that the newly obtained network can be applied to other fields similar to the source domain (i.e., the target domain). For example, a neural network suitable for semantic feature extraction in sunny scenes can be applied to semantic feature extraction tasks in foggy scenes or thunderstorm scenes through transfer learning. In the prior art, there are generally two solutions for transfer learning:

[0056] (1) Use part of the data and labels to train and adjust the trained network backend parameter layer (also known as the weight layer) according to needs, that is, use the fine-tuning method to achieve network migration.

[0057] (2) Use unlabeled data to train the trained network and use adversarial loss to adjust the network, that is, use unsupervised domain adaptation methods to achieve network migration.

[0058] However, the solution of achieving network migration by fine-tuning the network backend parameter layer requires the use of a large amount of labeled data for training, and the required large amount of data and labels are difficult to obtain. Therefore, in order to reduce the dependence on the large amount of data and labels required for training, it is necessary to use an unsupervised domain adaptive transfer learning solution.

[0059] The traditional unsupervised domain adaptive transfer learning solution is to transfer the neural network model from a single source domain to another single target domain. In order to accurately extract semantic features in various weather conditions, it is necessary to transfer the existing source weather domain network multiple times. For example, when the source weather domain is sunny, it is necessary to transfer the network once for each weather domain such as cloudy, rainy, foggy, etc. to adapt to the image data of various weather domains. In other words, it takes extremely high time and computing power costs to complete the adaptation of the network model to various weather conditions (which can also be understood as various scenes).

[0060] At the same time, as mentioned above, the traditional unsupervised domain adaptive transfer learning scheme needs to perform multiple migrations for various weather conditions. Each migration will obtain different network parameters for different weather domains, and storage space needs to be allocated separately for each network parameter. At the same time, additional training of the weather judgment network is required to call different network parameters according to different weather conditions, which requires a large amount of storage space, thereby increasing the hardware burden.

[0061] To sum up, in the related technologies, semantic feature extraction networks are used in video surveillance systems to assist in monitoring and early warning. It is necessary to migrate the semantic feature extraction networks suitable for a single weather domain to be suitable for multiple weather conditions. The existing transfer learning solutions have problems of high time cost, high computing power cost, and heavy hardware burden.

[0062] Based on this, in various embodiments of the present application, a semantic feature extraction model applicable to multiple target weather domains is constructed in the cloud; the image data of the source weather domain, the image data of at least two target weather domains including the source weather domain, the model for extracting the semantic features of the source weather domain monitoring data, the model for predicting the weather domain to which the image data belongs, and the model that can determine whether the input feature data comes from the source weather domain or the target weather domain are used to jointly train a semantic feature extraction model applicable to the monitoring data of multiple target weather domains, and the semantic feature extraction model applicable to the monitoring data of multiple target weather domains trained in the cloud is sent to the monitoring system for deployment. In this way, only one migration is needed to obtain a model applicable to multiple weather conditions, thereby realizing simultaneous migration to multiple target weather domains, reducing the time required for training, and reducing computing power costs; at the same time, one model can be applicable to multiple weather conditions, and only the parameters of one model need to be stored, thus saving storage space.

[0063] The present application embodiment provides an information processing method, which is applied to the cloud. Figure 1 As shown, the method includes:

[0064] Step 101: construct a first model;

[0065] Step 102: training the first model using the acquired first image data, second image data, second model, third model, and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes semantic features output by the second model and semantic features output by the first model;

[0066] Step 103: Send the trained first model to the monitoring system, wherein, for the image data to be identified in the target weather domain, the first data is matched with the second data, the first data comprises a first semantic feature and a corresponding weather domain label, the first semantic feature comprises a semantic feature extracted by the trained first model, and the second data comprises a second semantic feature and a corresponding weather domain label, the second semantic feature comprises a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

[0067] Among them, in actual application, the method of the embodiment of the present application is applied to a cloud device, such as a cloud server, etc. As long as it is a cloud device that can implement the solution of the embodiment of the present application, the embodiment of the present application does not limit this.

[0068] In actual application, the monitoring system in the embodiment of the present application can be set up as needed. The monitoring system can at least obtain monitoring data and extract semantic features from the monitoring data. The embodiment of the present application does not limit the specific implementation of the monitoring system.

[0069] For example, Figure 2 As shown, the monitoring system can be an outdoor video monitoring system, which specifically includes: video acquisition module, semantic feature extraction module, target recognition and segmentation module, local storage device, cloud server storage device, signal sending module, video stream protocol conversion module, display device and user terminal device. The video monitoring system can quickly and accurately complete the following tasks:

[0070] Live playback: The video data is collected in real time through the video acquisition module, the semantic features in the video data are extracted using the semantic feature extraction module, the extracted semantic features are processed using the target recognition and segmentation module to achieve target recognition and semantic segmentation, the processed video data is converted into a multi-channel video stream using the video stream protocol conversion module, and then the video stream data is played through the display device;

[0071] Monitoring and early warning: When the target recognition and segmentation module identifies the existence of situations in the target area, such as: intrusion of outsiders, collision of objects, and falling of people, the signal sending module will promptly send early warning information to the user terminal device to remind the management personnel;

[0072] Video playback: The original video captured by the video acquisition module and the video processed by the video processing algorithm module can be uploaded to the cloud or backed up to the local hard disk (i.e. local device storage). Managers can call local videos or download cloud videos at any time;

[0073] Remote control: Administrators can remotely control the video acquisition module through user terminal devices to change the acquisition angle, zoom, etc. At the same time, administrators can also modify files stored in local storage devices.

[0074] Here, it should be noted that when processing monitoring images in a monitoring system, after using the semantic feature extraction model, other models can be used to further process the extracted semantic features to jointly constitute a source weather domain network to meet the needs of actual applications. For example, a semantic segmentation model can be used to identify human body parts in semantic features and divide the area where the human body is located. When human body parts appear in an area pre-set as prohibited entry in the monitoring data, it can be determined that an illegal intrusion has occurred. The specific model used can be set according to the actual application needs, and the embodiments of the present application are not limited to this.

[0075] In actual application, in order to achieve accurate semantic feature extraction of image data in multiple weather domains, it is necessary to obtain a semantic feature extraction model suitable for multiple target weather domains through transfer learning. The semantic feature extraction model can extract semantic features of monitoring data of at least two target weather domains including the source weather domain.

[0076] Specifically, in order to achieve migration to multiple target weather domains at the same time, after semantic features are extracted from the image data, the semantic features and the corresponding weather domain labels are tensor spliced, wherein the weather domain labels correspond to the weather domains one-to-one and are used to identify the weather domain of the image data corresponding to the semantic features (which can also be understood as the weather conditions corresponding to the image data). The specific implementation of the tensor splicing may include: when the weather domain label is a one-dimensional vector and the semantic feature is a two-dimensional vector, the weather domain label can be encoded into a corresponding two-dimensional vector and spliced ​​with the semantic feature. The data obtained after splicing can be called a joint feature or a joint feature map.

[0077] At this time, in order to achieve simultaneous migration to multiple target weather domains, the first model needs to be trained, so as to achieve: the joint features (i.e., the first data) obtained by tensor splicing of the semantic features extracted by the first model and the corresponding weather domain labels are matched with the joint features (i.e., the second data) obtained by tensor splicing of the semantic features extracted by the second model and the corresponding weather domain labels when the weather domain of the image data is replaced with the source weather domain. When the first data matches the second data, the trained first model can be used to extract the semantic features of the multi-target weather domain image data, and at the same time, the weather domain labels contained in the first data can be used to determine the weather domains corresponding to the semantic features, thus achieving simultaneous migration from the source domain to multiple target domains.

[0078] Here, it should be noted that each of the at least two target weather domains is in a parallel and non-overlapping relationship, such as rainy days, cloudy days, snowy days, sunny days, etc. The target weather domain can be set as needed, such as determined according to the weather conditions in the video image data collected according to the actual monitoring scene. The embodiment of the present application is not limited to this.

[0079] More specifically, the process of transfer learning includes: using an existing model for extracting semantic features of source weather domain image data (i.e., the second model) for migration to obtain a model for extracting semantic features of multiple target weather domain image data (i.e., the first model), wherein the second model is already trained, and specifically, can be trained using monitoring data of the source weather domain (i.e., the second image data) before transfer learning.

[0080] Based on this, in one embodiment, before step 101, the method may further include:

[0081] The second model is trained using the second image data.

[0082] Here, it should be noted that the structure of the second model can be selected based on the actual application. For example, the second model can be designed according to the actual application needs, or Unet++, SegNet, ResNet and other excellent segmentation network models can be used as the basic network model for design, and adjusted according to the actual application needs. The embodiments of the present application are not limited to this.

[0083] Exemplarily, when it is necessary to process the semantic features of the image data extracted by the second model to obtain semantic segmentation in an actual application, the UNet++ segmentation network can be used as the basic network to design the second model. Specifically, the UNet++ segmentation network can be adjusted as follows: the UNet++ segmentation network consists of two parts. The first part is a U-type encryption-decryption (encoder-decoder) network, which extracts shallow features and deep features respectively, and splices the shallow features and the deep features together; the second part is an n-layer convolutional feature layer, where n is the number of classification categories, which is used to segment the output of the network; when designing the second model, the structure of the first part in the UNet++ segmentation network is used as the structure of the second model.

[0084] Here, it should be noted that when using a segmentation network such as SegNet, ResNet, etc. as a basic network model to design the second model, the above design method can also be used.

[0085] In actual application, transfer learning is performed based on the second model to obtain a first model applicable to multiple target weather domains, the first model is used to extract semantic features of the image data to be identified, and the semantic features are tensor-joined with the corresponding weather domain labels to obtain joint features, so as to distinguish the image data of multiple different target weather domains. When other models are used to further process the semantic features corresponding to multiple target weather domains (such as semantic segmentation), the input of the other models includes the joint features.

[0086] In practical application, the training of the second model may specifically include: training the second model using supervised learning. For example, Figure 3As shown, when the monitoring system includes a video acquisition module, a semantic feature extraction module, and a target recognition and segmentation module, the monitoring system acquires video data in real time through the video acquisition module, uses the semantic feature extraction module to extract semantic features in the video data, and uses the target recognition and segmentation module to process the extracted semantic features to achieve target recognition and semantic segmentation. For the monitoring system, the semantic feature extraction module (i.e., the second model, which can also be understood as performing source domain network training) can be trained in the cloud. First, a plurality of source weather domain images (i.e., the second image data) for training and corresponding manually marked semantic segmentation labels are obtained, and then the semantic feature extraction module (specifically, it can be a source weather domain feature generator) is used to extract the semantic features of the source weather domain image to obtain a source weather domain feature map, and the source weather domain feature map and the source weather domain label converted by word vector are tensor-joined to obtain a source weather domain joint feature map, and the source weather domain joint feature map is input into the target recognition and segmentation module to obtain a semantic segmentation result, and the semantic segmentation result is compared with the manually marked semantic segmentation label, and the semantic segmentation loss L is calculated by cross entropy. seg ; Based on semantic segmentation loss L seg The semantic feature extraction module is trained and optimized to obtain a second model that can be used to extract semantic features of the second image data.

[0087] In actual application, the structure of the first model can be set as needed, as long as the model structure can realize semantic feature extraction of image data. In order to build the first model more conveniently and quickly, and avoid negative transfer in transfer learning to achieve better transfer learning effect, the structure of the first model can use the same structure as the second model, and use the model parameters of the second model as the initialization parameters of the first model.

[0088] Based on this, in one embodiment, the specific implementation of step 101 may include:

[0089] The first model is constructed using the second model.

[0090] Specifically, in one embodiment, using the second model to construct the first model includes:

[0091] Determine the structure of the first model using the structure of the second model;

[0092] The parameters of the second model are determined as the parameters of the first model.

[0093] That is, the first model has the same structure as the second model, and the parameters of the first model are initialized using the parameters of the second model. This reduces the time to build the first model, facilitates transfer learning, and also reduces the negative transfer caused by the different structures of the first model and the second model during the transfer process.

[0094] In actual application, before training the first model, it is necessary to obtain monitoring data (first image data) of at least two target weather domains and monitoring data (second image data) of the source weather domain, wherein the monitoring data under various weather conditions is collected in the actual application scenario by the video acquisition device in the monitoring system and uploaded to the cloud, so that the cloud can obtain the first image data. The embodiment of the present application does not limit the method of obtaining the first image data. The cloud can obtain the second image data through a historical database or a network public database. The embodiment of the present application does not limit the method of obtaining the second image data.

[0095] In actual application, for the image data to be identified in the target weather domain, in order to achieve the matching of the first data with the second data, that is, the first data with the second data, the fourth model can be constructed. The input data of the fourth model includes the first data and the second data. The fourth model is used to distinguish the input data, that is, to judge whether the input data belongs to the source weather domain (which can also be understood as belonging to the second data) or the target weather domain (which can also be understood as belonging to the first data). In this way, the fourth model forms a competitive relationship with the first model, and the weather domain label is used as a condition (which can also be understood as supervision information) to form CGAN. On the one hand, the training purpose of the first model is to make the output semantic features of the first data obtained after being labeled with the weather domain label match the semantic features output by the second model after being labeled with the weather domain label, so that the fourth model cannot accurately distinguish the first data from the second data; on the other hand, the training purpose of the fourth model is to distinguish the first data from the second data as accurately as possible. Through this competitive training process, the first model continuously adjusts the strategy of extracting semantic features to make the first data and the second data more matched. At the same time, the fourth model also continuously adjusts its own distinguishing ability to better distinguish the first data from the second data.

[0096] Here, it should be noted that the weather domains of the image data corresponding to the second data are all source weather domains, and the weather domains of the image data corresponding to the first data include the source weather and as a target weather domain. When the weather domain of the image data corresponding to the joint feature input into the fourth model is the source weather domain, the fourth model can still be used to determine whether the joint feature is the first data or the second data. In other words, the semantic features of the image data of the source weather domain are extracted by the first model and the second model respectively, and after being labeled with the weather domain labels, the obtained joint features can be distinguished by the fourth model. The reason is that during the training process, the ability to extract the semantic features of the image data of the source weather domain through the first model will be affected by the image data of other target weather domains outside the source weather domain, resulting in a negative transfer effect, resulting in the semantic features of the image data of the source weather domain extracted by the first model cannot obtain the same result as the semantic features of the image data of the source weather domain extracted by the second model.

[0097] Specifically, compared with generative adversrial networks (GAN) that do not use supervised information, CGAN can increase the controllability of transfer learning, that is, using weather domain labels to annotate semantic features can guide the optimization of the fourth model and the first model to achieve better optimization results.

[0098] Based on this, in one embodiment, the method may further include:

[0099] For each image data in the first image data, the predicted weather domain is used as the weather domain label of the image data; wherein,

[0100] For each image data in the second image data, the source weather domain is a weather domain label of the image data.

[0101] Specifically, in one embodiment, for each image data in the second image data, a weather domain label of the image data is preset.

[0102] For example, Figure 5 As shown, the features of the fourth model input for training the first model include two joint features (i.e., the first data and the second data), which correspond to the outputs of the first model and the second model respectively. For the semantic features output by the second model, since the second model is used to extract the semantic features of the second image data (the monitoring data of the source weather domain), it is possible to predetermine the weather domain label corresponding to the second data as the weather domain label corresponding to the source weather domain, convert the source weather domain label into a word vector, and perform tensor splicing of the converted word vector and the semantic features output by the second model to obtain the second data as an input of the fourth model;

[0103] For the semantic features output by the first model, the first model is used to extract the semantic features of the first image data (including monitoring data of at least two target weather domains). For each image data in the first image data, the third model can predict the weather domain to which each image data belongs. The predicted weather forecast result (specific weather domain, such as rainy days, foggy days, snowy days, etc.) is used as the weather domain label, the weather forecast result is converted into a word vector, and the converted word vector is tensor-concatenated with the semantic features output by the first model to obtain the first data as another input of the fourth model.

[0104] In this way, adding weather domain labels during model training and guiding the output of the first model to match the output of the second model can further optimize the parameters of the fourth model and the first model, so that the first data corresponding to multiple target weather domains and the second data corresponding to the source weather domain are more matched, thereby realizing the migration of the network model from the source weather domain to multiple target weather domains.

[0105] Based on this, in one embodiment, the specific implementation of step 102 may include:

[0106] For each image data in the first image data, extracting semantic features of the image data using the first model;

[0107] Predicting the weather domain to which the image data belongs by using the third model and the semantic features of the image data;

[0108] The weather domain to which the image data corresponding to the semantic feature belongs is determined by using the semantic features of the image data and the corresponding weather domain, as well as the third data and the fourth model, wherein the third data includes the semantic features of the image data and the source weather domain obtained by using the second model and the second image data; wherein,

[0109] The parameters of the first model, the third model and the fourth model are optimized by the loss function.

[0110] Specifically, in one embodiment, optimizing the parameters of the first model, the third model, and the fourth model by using a loss function includes:

[0111] Optimizing the parameters of the first model and the third model simultaneously through a first loss function so that the predicted weather domain is the same as the weather domain to which the image data actually belongs;

[0112] Optimizing the parameters of the fourth model by using the second loss function so that the weather domain predicted by the fourth model is the same as the weather domain to which the image data actually belongs;

[0113] The parameters of the first model are optimized by a third loss function so that the first data matches the second data.

[0114] In practical application, the third model can be called a weather classifier. For example, Figure 4 As shown, the first image data X is extracted using the first model t The characteristics of the target weather domain are obtained by Lt , map the target weather domain feature M Lt As the weather classifier C w Input, get the weather forecast result P w , the weather forecast result P w With weather domain label Y w For comparison, the multi-classification loss L is calculated by cross entropy cls ; Based on multi-classification loss L cls The target weather domain feature generator G t and weather classifier C w Train and optimize. In this way, the trained weather classifier C w It is possible to accurately determine the weather type corresponding to the feature map of the input image data.

[0115] It should be noted that the specific implementation of the third model can be determined as needed and can be any model that can classify weather through semantic features, and the embodiments of the present application are not limited to this.

[0116] In practical applications, the fourth model can be called a discriminator. Figure 6 As shown, the first data and the second data are discriminated by the discriminator D to obtain the GAN loss (i.e. the second loss function) and GAN loss (i.e., the third loss function), where GAN loss is used The parameters of the discriminator D are optimized so that the discriminator D can judge whether the joint feature input to the discriminator belongs to the first data or the second data as accurately as possible, that is, to distinguish whether the image data corresponding to the semantic feature belongs to the source weather domain or the target weather domain; at the same time, the GAN loss is used The parameters of the first model are optimized so that the first data corresponding to the first model matches the second data corresponding to the second model, making it difficult for the discriminator D to make the above distinction, thereby achieving the purpose of "deceiving" the discriminator D.

[0117] Exemplarily, the discriminator may be a feature-level discriminator, which uses a fully connected layer to output a logical prediction result, such as Figure 7 As shown in the figure, the feature map of the input discriminator is connected through two layers of ReLU activation function, and finally the normalized exponential function (softmax) is used to calculate the logical prediction result and output the discrimination result.

[0118] It should be noted that the output of the discriminator represents whether the joint features of the input belong to the first data or the second data, but does not represent which specific target weather domain the input image comes from. For example, the output "0" of the discriminator represents that the joint features of the input belong to the second data, that is, from the source weather domain network, and the output "1" represents that the joint features of the input belong to the first data, that is, from the target weather domain network.

[0119] Here, the number of iterations for optimizing the model using the loss function can be set as needed. In each optimization process, the first loss function, the second loss function and the third loss function can be used sequentially to optimize the corresponding model. The embodiment of the present application does not limit this.

[0120] It should be noted that the specific implementation of the fourth model can be determined as needed, and the embodiment of the present application does not limit this. The specific implementation of the discriminator can also be designed as needed, such as adding a convolution module, etc., and the embodiment of the present application does not limit this.

[0121] In actual application, after training, the first data corresponding to the first model matches the second data corresponding to the second model. For example, when sunny days are used as the source weather domain, and cloudy days, sunny days, and rainy days are used as the three target weather domains, the first data obtained by the same monitoring system for the same scene on cloudy days, rainy days, and sunny days after the semantic features extracted by the trained first model are annotated with weather domain labels, and the second data obtained by the semantic features extracted by the second model on sunny days for the same scene are annotated with weather domain labels. Therefore, other models that have been trained with image data of sunny days (source weather domain) can be directly used to further process the first data corresponding to the first model to obtain processing results for image data of multiple weather domains. In other words, the trained first model can realize accurate semantic feature extraction under changeable weather conditions, and can analyze the monitored area around the clock, avoiding the situation where dangerous factors and abnormalities cannot be detected due to special weather, and better realizing the auxiliary security function.

[0122] Here, it should be noted that in order to reduce time cost and computing power cost, other models in the source weather domain network do not need to be retrained.

[0123] In actual application, the trained first model needs to be sent to the monitoring system for local deployment, and the sending method can be achieved through wired transmission or wireless transmission, etc., as long as the sending method can implement the embodiment of the present application, the embodiment of the present application does not limit this.

[0124] Specifically, when monitoring images are processed in actual applications, if other models are included after extracting semantic features, a complete image data processing network can be built in the cloud based on the actual business and based on the trained first model. For example, a semantic segmentation network including a semantic feature extraction module and a semantic segmentation module can be built for the business task of semantic segmentation.

[0125] Based on this, in one embodiment, before step 103, the method may further include:

[0126] Based on the trained first model and the service, an image data processing network is constructed.

[0127] Accordingly, the specific implementation of step 103 may include:

[0128] The image data is processed and sent to a monitoring system.

[0129] In other words, the monitoring system can be directly deployed locally and quickly based on the received image data processing network to realize the processing business of monitoring data and save time and computing power costs in the local monitoring system.

[0130] The solution provided by the embodiment of the present application is to build a first model in the cloud; train the first model using the acquired first image data, second image data, second model, third model, and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, and the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model; send the trained first model to the monitoring system, wherein, for the image data to be identified in the target weather domain, the first data is matched with the second data, the first data includes a first semantic feature and a corresponding weather domain label, the first semantic feature includes a semantic feature extracted by the trained first model, the second data includes a second semantic feature and a corresponding weather domain label, and the second semantic feature includes a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced with the source weather domain. The solution provided by the embodiment of the present application is to build a semantic feature extraction model applicable to multiple target weather domains in the cloud; use image data of a source weather domain, image data of at least two target weather domains including the source weather domain, a model for extracting semantic features of monitoring data of the source weather domain, a model for predicting the weather domain to which the image data belongs, and a model that can determine whether the input feature data comes from the source weather domain or the target weather domain to jointly train a semantic feature extraction model applicable to monitoring data of multiple target weather domains; and send the semantic feature extraction model applicable to monitoring data of multiple target weather domains trained in the cloud to the monitoring system for deployment. In this way, only one migration is needed to obtain a model applicable to multiple weather conditions, thereby realizing simultaneous migration to multiple target weather domains, reducing the time required for training, and reducing computing power costs; at the same time, one model can be applicable to multiple weather conditions, and only the parameters of one model need to be stored, thus saving storage space.

[0131] The present application is further described in detail below in conjunction with application examples.

[0132] In this application example, information processing in an intelligent monitoring management system is used as an example to illustrate. This application example proposes an intelligent monitoring management system based on multi-domain transfer learning to cope with changing weather. Figure 8As shown, the system includes: a monitoring system and a cloud server. Among them, the monitoring system is composed of a monitoring acquisition device, a data processing device, a human-computer interaction device, a storage device and a management central control device. The monitoring acquisition device inputs the collected monitoring data into the data processing device, and a semantic segmentation network based on a neural network is deployed in the data processing device to process the monitoring data, specifically including: extracting semantic features in the monitoring data using a semantic feature extraction model, processing the semantic features using a semantic segmentation model, obtaining semantic segmentation results, and transmitting them to the human-computer interaction device and the storage device for real-time display and storage. The semantic segmentation results represent the scene conditions in the monitoring data. When the scene conditions are dangerous factors or abnormal conditions, an early warning prompt will be given to the management central control device, and the management central control device can control the monitoring acquisition device to perform operations such as angle change and zoom. The human-computer interaction device can access the monitoring data stored in the storage device at any time and call historical monitoring data for playback.

[0133] The cloud server is responsible for the transfer learning task of the semantic feature extraction model in the monitoring system. The monitoring system no longer needs to perform network training locally, which can save computing power, manpower and time costs.

[0134] In the cloud server, this application embodiment also proposes a semi-supervised domain adaptive information processing method for simultaneous migration from a single weather domain to a multi-target weather domain based on CGAN adversarial loss, such as Fig. 9 As shown, the method includes:

[0135] Step 901: training a source weather domain network;

[0136] The source weather domain network is trained using supervised learning. Specifically, Fig.10 As shown in the figure, the sunny day is taken as the source weather domain. The image data corresponding to the source weather domain has good weather conditions and clear and complete features. First, the image data X of the source weather domain is obtained. s (i.e., the second image data), weather domain label Y w (The weather domain label here is the source weather domain, i.e. sunny day) and the manually labeled semantic segmentation label Y s ; Afterwards, the source weather domain image data X s By using the source weather domain feature generator G s (i.e., the second model) generates the source weather domain feature map M Ls At the same time, the weather domain label Y w By converting the word vector into a word vector, the word vector is compared with the source weather domain feature map M Ls Perform tensor splicing to obtain the source weather domain joint feature map M Gs , the source weather domain joint feature map M GsAs the input of the source weather domain pixel-level classifier C (i.e., semantic segmentation model), the semantic segmentation prediction result P is obtained. s ; The semantic segmentation prediction result P s and semantic segmentation label Y s For comparison, the semantic segmentation loss L is calculated by cross entropy seg ; Based on semantic segmentation loss L seg The source weather domain feature generator G s The parameters of the pixel-level classifier C in the source weather domain are trained and optimized to obtain a semantic segmentation network suitable for the source weather domain.

[0137] Step 902: Multi-target weather domain migration;

[0138] The multi-objective weather domain network is trained using a semi-supervised learning method. In view of the uncontrollable shortcomings of GAN itself, CGAN is used to add supervision information to the training, such as Fig.11 As shown, guide GAN to generate.

[0139] All weather conditions including the source weather domain (sunny day) are taken as the target weather domain, and the image data X obtained under all weather conditions is taken as t (i.e., the first image data) and the manually marked weather domain label Y w And a small amount of target weather domain semantic segmentation labels Y t Different from the source weather domain network training, in the process of multi-target weather domain migration, a small amount of semantic segmentation labels Y t It does not participate in the training of the target weather domain network and is only used to detect the effect of semantic segmentation using the multi-target weather domain network obtained after migration.

[0140] In order to reduce the target weather domain feature generator G in the migration process t The generated negative transfer, target weather domain feature generator G t (i.e. the first model) uses the source weather domain feature generator G s The same network model structure, and load the trained source weather domain feature generator G s The network parameters of are used as initialization parameters, and the source weather domain feature generator G is not updated during the training process. s network parameters.

[0141] The training is mainly divided into three steps, corresponding to L cls , and Three types of losses:

[0142] Step 9021: Multi-target weather domain image X t After the target weather domain feature generator G tGenerate target weather domain feature map M Lt , map the target weather domain feature M Lt Input weather classifier C w (i.e. the third model) to obtain the weather forecast P w , and compare the obtained weather forecast results with the weather domain label Y w For comparison, use cross entropy to calculate the multi-classification loss L cls , use the calculated loss to the target weather domain feature generator G t and weather classifier C w In this way, a weather classifier that can accurately judge the weather type of the input image data can be obtained, and the target domain feature generator G can be optimized at the same time. t ;

[0143] Step 9022: Transform the source weather domain image X s By using the source weather domain feature generator G s Generate source weather domain feature map M Ls , and then converted to word vector weather domain label Y w (Here the source weather domain label is sunny) tensor splicing is performed to obtain the source weather domain joint feature map M Gs ; Map the target weather domain feature M Lt and the weather forecast P converted to word vector w Perform tensor concatenation to obtain the target domain joint feature map M Gt ; Finally, the source weather domain joint feature map M Gs and the target weather domain joint feature map M Gt Input discriminator D (i.e. the fourth model), using GAN loss The discriminator D is optimized and trained. By minimizing the loss Optimize the parameters of the discriminator D and enhance its judgment ability so that it can correctly judge whether the input features come from the source weather domain or the target weather domain;

[0144] Step 9023: Map the target weather domain joint feature map M Gt Input discriminator D uses GAN loss For the target weather domain feature generator G t Optimize and train. By minimizing the loss Optimize the weather domain feature generator G t The parameters of the target weather domain joint feature map M of the input discriminator D Gt Approaching the joint feature map M of the source weather domain Gs , to achieve the purpose of "deceiving" the discriminator;

[0145] The network parameters are further optimized through iterative training, and steps 9021, 9022, and 9023 are performed alternately in each iteration.

[0146] Step 903: Construct a multi-weather semantic segmentation network.

[0147] After the multi-weather semantic segmentation network is constructed, it is sent to the monitoring system for construction, e.g. Fig.12 As shown, the target weather domain feature generator G t , Weather Classifier C w and the source domain pixel-level classifier C. The migration reduces the difference between the features generated by the source weather domain network and the multi-target weather domain network, so that the semantic features generated by the target weather domain can be recognized by the source weather domain pixel-level classifier, thereby generating semantic segmentation prediction results.

[0148] After the migration is completed, the target weather domain feature generator G t With weather classifier C w The jointly generated target weather domain joint feature map M Gt Approaching the source weather domain feature generator G s and weather domain label Y w The jointly generated source weather domain joint feature map M Gs , so the pixel-level classifier C in the source weather domain can accept the joint feature map M in the target weather domain Gt As input, and complete the multi-target weather domain image X t semantic segmentation task.

[0149] From the above description, it can be seen that the method provided by this application example first trains the semantic segmentation network of the source weather domain, and then builds a semantic feature extraction model suitable for multiple target weather domains in the cloud; using image data Xs and image data X t , Source weather domain feature generator G s , Weather Classifier C w And the discriminator D can jointly train the target weather domain feature generator G t , and based on the target weather domain feature generator G trained in the cloud t Construct a multi-weather semantic segmentation network, and finally send the multi-weather semantic segmentation network to the data processing equipment of the monitoring system for deployment. In this way, only one migration is needed to obtain a semantic segmentation network suitable for multiple weather conditions, and simultaneous migration to multiple target weather domains is realized, which reduces the time required for training and reduces computing power costs. At the same time, one network can be applicable to multiple weather conditions, and only the parameters of one network need to be stored, thus saving storage space.

[0150] From the above description, it can be seen that this application example provides an intelligent monitoring and management system based on multi-domain transfer learning to cope with changing weather, such as Fig.13 As shown, the intelligent monitoring management system can be applied in the intelligent monitoring management platform, and the monitoring system network parameters are trained in the cloud server and deployed locally and quickly in the local hardware platform, saving the local deployment computing cost. Specifically, the source weather domain network is trained in the cloud using sunny monitoring images and corresponding manually marked semantic segmentation labels, and then the source weather domain network is migrated using monitoring images in various weather conditions to obtain a target weather domain network suitable for various weather conditions, and then the trained target weather domain network is sent to the local hardware platform for installation to achieve local deployment, which can realize accurate semantic feature extraction and target recognition and segmentation of monitoring images in various weather conditions obtained by the video acquisition module, thereby better assisting security work.

[0151] In order to implement the information processing method of the embodiment of the present application, the embodiment of the present application also provides an information processing device, which is set on the cloud, such as Fig.14 As shown, the device comprises:

[0152] A construction module 1401 is used to construct a first model;

[0153] The training module 1402 is used to train the first model using the acquired first image data, second image data, second model, third model, and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model;

[0154] The sending module 1403 is used to send the trained first model to the monitoring system, wherein, for the image data to be identified in the target weather domain, the first data is matched with the second data, the first data includes a first semantic feature and a corresponding weather domain label, the first semantic feature includes a semantic feature extracted by the trained first model, and the second data includes a second semantic feature and a corresponding weather domain label, the second semantic feature includes a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

[0155] In one embodiment, the training module 1402 is specifically used to extract semantic features of image data using the first model for each image data in the first image data; predict the weather domain to which the image data belongs using the third model and the semantic features of the image data; determine the weather domain to which the image data corresponding to the semantic features belongs using the semantic features of the image data and the corresponding weather domain, as well as the third data and the fourth model, wherein the third data includes the semantic features of the image data and the source weather domain obtained using the second model and the second image data; wherein the parameters of the first model, the third model and the fourth model are optimized by the loss function.

[0156] In one embodiment, the training module 1402 is specifically used to simultaneously optimize the parameters of the first model and the third model through a first loss function so that the predicted weather domain is the same as the weather domain to which the image data actually belongs; optimize the parameters of the fourth model through a second loss function so that the weather domain predicted by the fourth model is the same as the weather domain to which the image data actually belongs; optimize the parameters of the first model through a third loss function so that the first data matches the second data.

[0157] In one embodiment, the training module 1402 is also used to use the predicted weather domain as the weather domain label of the image data for each image data in the first image data; wherein, for each image data in the second image data, the source weather domain is the weather domain label of the image data.

[0158] In one embodiment, for each image data in the second image data, a weather domain tag of the image data is preset.

[0159] In one embodiment, the construction module 1401 is specifically configured to construct the first model using the second model.

[0160] In one embodiment, the construction module 1401 is specifically configured to determine the structure of the first model using the structure of the second model; and determine the parameters of the second model as the parameters of the first model.

[0161] In actual application, the sending module 1403 can be implemented by a communication interface in an information processing device; the building module 1401 and the training module 1402 can be implemented by a processor in the information processing device.

[0162] It should be noted that: when the information processing device provided in the above embodiment performs information processing, only the division of the above program units is used as an example. In actual applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device is divided into different program units to complete all or part of the processing described above. In addition, the information processing device provided in the above embodiment and the information processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0163] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a cloud device, such as Fig.15 As shown, the cloud device 1500 includes:

[0164] Communication interface 1501, capable of interacting with the monitoring system;

[0165] A processor 1502, connected to the communication interface 1502, to interact with the monitoring system, and used to execute the method provided by one or more of the above technical solutions when running the computer program;

[0166] The computer program is stored in the memory 1503 .

[0167] Specifically, the communication interface 1501 is used to send the trained first model to the monitoring system, wherein, for the image data to be identified in the target weather domain, the first data is matched with the second data, the first data includes a first semantic feature and a corresponding weather domain label, the first semantic feature includes a semantic feature extracted by the trained first model, and the second data includes a second semantic feature and a corresponding weather domain label, the second semantic feature includes a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced with the source weather domain;

[0168] The processor 1502 is configured to:

[0169] Build the first model;

[0170] The first model is trained using the acquired first image data, second image data, second model, third model and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model.

[0171] In one embodiment, the processor 1502 is configured to:

[0172] For each image data in the first image data, the first model is used to extract the semantic features of the image data; the third model and the semantic features of the image data are used to predict the weather domain to which the image data belongs; the semantic features of the image data and the corresponding weather domain, as well as the third data and the fourth model are used to determine the weather domain to which the image data corresponding to the semantic features belongs, the third data containing the semantic features of the image data and the source weather domain obtained using the second model and the second image data; wherein the parameters of the first model, the third model and the fourth model are optimized by a loss function.

[0173] In one embodiment, the processor 1502 is configured to:

[0174] The parameters of the first model and the third model are optimized simultaneously through the first loss function, so that the predicted weather domain is the same as the weather domain to which the image data actually belongs; the parameters of the fourth model are optimized through the second loss function, so that the weather domain predicted by the fourth model is the same as the weather domain to which the image data actually belongs; the parameters of the first model are optimized through the third loss function, so that the first data matches the second data.

[0175] In one embodiment, the processor 1502 is configured to:

[0176] For each image data in the first image data, the predicted weather domain is used as the weather domain label of the image data; wherein, for each image data in the second image data, the source weather domain is the weather domain label of the image data.

[0177] In one embodiment, for each image data in the second image data, a weather domain label of the image data is preset.

[0178] In one embodiment, the processor 1502 is configured to:

[0179] The first model is constructed using the second model.

[0180] In one embodiment, the processor 1502 is configured to:

[0181] The structure of the first model is determined by using the structure of the second model; and the parameters of the second model are determined as the parameters of the first model.

[0182] It should be noted that the specific processing process of the communication interface 1501 and the processor 1502 can be understood by referring to the above method.

[0183] Of course, in actual application, the various components in the cloud device 1500 are coupled together through the bus system 1504. It can be understood that the bus system 1504 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.14 Various buses are labeled as bus system 1504.

[0184] The memory 1503 in the embodiment of the present application is used to store various types of data to support the operation of the cloud device 1500. Examples of such data include: any computer program used to operate on the cloud device 1500.

[0185] The method disclosed in the above embodiment of the present application can be applied to the processor 1502, or implemented by the processor 1502. The processor 1502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 1502 or an instruction in the form of software. The above-mentioned processor 1502 may be a general-purpose processor, a digital signal processor (DSP, DigitalSignal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 1502 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 1503, and the processor 1502 reads the information in the memory 1503 and completes the steps of the above method in combination with its hardware.

[0186] It can be understood that the memory (memory 1503) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0187] In an exemplary embodiment, the cloud device 1500 can be implemented by one or more application-specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field-programmable gate array (FPGA), general-purpose processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0188] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a memory 1503 storing a computer program, and the above-mentioned computer program can be executed by the processor 1502 of the cloud device 1500 to complete the steps of the aforementioned model training method, or to complete the steps of the aforementioned path determination method. The computer-readable storage medium can be ROM, PROM, EPROM, EEPROM, FRAM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM; the magnetic surface storage can be a disk storage or a tape storage.

[0189] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0190] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0191] The above is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An information processing method, It is characterized in that Applied in the cloud, including: Build the first model; The first model is trained using the acquired first image data, second image data, second model, third model, and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used to extract semantic features of the second image data, the third model is used to predict the weather domain to which the first image data belongs, the fourth model is used to determine whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model; The trained first model is sent to the monitoring system, wherein, for image data to be identified in the target weather domain, the first data is matched with the second data, the first data comprises a first semantic feature and a corresponding weather domain label, the first semantic feature comprises a semantic feature extracted by the trained first model, and the second data comprises a second semantic feature and a corresponding weather domain label, the second semantic feature comprises a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

2. The method according to claim 1, It is characterized in that The method of training the first model by using the first image data, the second image data, the second model, the third model, and the fourth model of at least two target weather domains obtained includes: For each image data in the first image data, extracting semantic features of the image data using the first model; Predicting the weather domain to which the image data belongs by using the third model and the semantic features of the image data; The weather domain to which the image data corresponding to the semantic feature belongs is determined by using the semantic features of the image data and the corresponding weather domain, as well as the third data and the fourth model, wherein the third data includes the semantic features of the image data and the source weather domain obtained by using the second model and the second image data; wherein, The parameters of the first model, the third model and the fourth model are optimized by the loss function.

3. The method according to claim 2, It is characterized in that Optimizing the parameters of the first model and the third model simultaneously through a first loss function so that the predicted weather domain is the same as the weather domain to which the image data actually belongs; Optimizing the parameters of the fourth model by using the second loss function so that the weather domain predicted by the fourth model is the same as the weather domain to which the image data actually belongs; The parameters of the first model are optimized by a third loss function so that the first data matches the second data.

4. The method according to claim 1, It is characterized in that The method further comprises: For each image data in the first image data, the predicted weather domain is used as the weather domain label of the image data; wherein, For each image data in the second image data, the source weather domain is a weather domain label of the image data.

5. The method according to claim 4, It is characterized in that For each image data in the second image data, the weather domain label of the image data is preset.

6. The method according to claim 1, It is characterized in that The constructing of the first model comprises: The first model is constructed using the second model.

7. The method according to claim 6, It is characterized in that The using the second model to construct the first model includes: Determine the structure of the first model using the structure of the second model; The parameters of the second model are determined as the parameters of the first model.

8. An information processing device, It is characterized in that include: A construction module, used for constructing a first model; a training module, used for training the first model by using the acquired first image data, second image data, second model, third model and fourth model; the first image data includes monitoring data of at least two target weather domains, the second image data includes monitoring data of a source weather domain, the at least two target weather domains include the source weather domain, the second model is used for extracting semantic features of the second image data, the third model is used for predicting the weather domain to which the first image data belongs, the fourth model is used for judging whether the input data belongs to the source weather domain or the target weather domain, and the input data includes the semantic features output by the second model and the semantic features output by the first model; A sending module is used to send the trained first model to a monitoring system, wherein, for image data to be identified in a target weather domain, the first data is matched with the second data, the first data includes a first semantic feature and a corresponding weather domain label, the first semantic feature includes a semantic feature extracted by the trained first model, and the second data includes a second semantic feature and a corresponding weather domain label, the second semantic feature includes a semantic feature extracted by the second model when the weather domain of the image data to be identified is replaced by the source weather domain.

9. A cloud device, It is characterized in that include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images