Model training method, astronomical image-based retrieval method, astronomical image-based retrieval device and medium

Abstract

The invention discloses a model training method, an astronomical image-based retrieval method and apparatus, and a medium, after an astronomical image is obtained, a regression label and a classification label corresponding to the astronomical image are determined, and then the astronomical image is input into a to-be-trained classification model, so that a feature extraction layer in the classification model is used to extract a feature of the astronomical image; determining image features corresponding to the astronomical image, inputting the astronomical image into a classification layer in the classification model to obtain a classification result corresponding to the astronomical image, inputting the image features into a regression layer in the classification model to obtain a regression result corresponding to the astronomical image, and according to a deviation between the classification result and a classification label, determining a classification result corresponding to the astronomical image; and determining a total loss value corresponding to the astronomical image according to the regression result and the deviation between the regression result and the regression label, so as to at least train a feature extraction layer in the classification model according to the total loss value, thereby realizing accurate expression of the image features of the astronomical image.

Description

Technical Field

[0001] The present application relates to the fields of computer technology and astronomy, and in particular to a model training method, a retrieval method based on astronomical images, a device and a medium. Background Art

[0002] With the continuous development of astronomy and the gradual improvement of observation technology, various advanced telescopes are used to collect images of outer space. For example, the well-known "Hubble Telescope", "James Webb Telescope", etc. They can capture various galaxy features of different sizes in outer space and generate high-precision astronomical images, and astronomers can use these astronomical images to study various astronomical phenomena or celestial bodies.

[0003] In the process of astronomers' astronomical research, sometimes it is necessary to retrieve images with similar features from the stored astronomical images, so as to summarize and study the characteristics of celestial bodies with similar features through these retrieved images. However, due to the extremely large number of stored astronomical images, astronomers may not be able to retrieve the required images from the vast amount of astronomical images, which brings certain difficulties to astronomical research. Summary of the Invention

[0004] Embodiments of the present application provide a model training method, a retrieval method based on astronomical images, a device and a medium to partially solve the above problems existing in the prior art.

[0005] The present application adopts the following technical solutions: Embodiments of the present application provide a model training method, including: Obtain astronomical images; Determine the regression label and classification label corresponding to the astronomical image, where the regression label is used to represent the characteristics of the celestial body or galaxy corresponding to the astronomical image; Input the astronomical image into a classification model to be trained, so as to determine the image features corresponding to the astronomical image through the feature extraction layer in the classification model, and input the image features into the classification layer in the classification model to obtain the classification result corresponding to the astronomical image, and input the image features into the regression layer in the classification model to obtain the regression result corresponding to the astronomical image; Determine the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, so as to train at least the feature extraction layer in the classification model according to the total loss value.

[0006] Optionally, the regression label includes the brightness of the celestial body corresponding to the celestial image; Determine the regression label corresponding to the astronomical image, specifically including: Judge whether the center point of the astronomical image is within the galaxy contour; If so, determine the brightness of the celestial body corresponding to the astronomical image according to the pixel gray values within the galaxy contour in the astronomical image.

[0007] Optionally, before inputting the astronomical image into the classification model to be trained, the method further includes: Adopt a preset preprocessing method to perform image preprocessing on the celestial body image to obtain a preprocessed image, and the preprocessing method includes at least one of: adjusting the image size, image enhancement, adding noise, and image rotation; Input the astronomical image into the classification model to be trained, specifically including: Input the preprocessed image into the classification model to be trained.

[0008] Optionally, before determining the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, the method further includes: Through the feature extraction layer, determine the similarity between the image features corresponding to the astronomical image and the image features corresponding to other astronomical images; According to the similarity, determine the positive sample image corresponding to the astronomical image and the negative sample image corresponding to the astronomical image, where the similarity between the image features corresponding to the astronomical image and the image features corresponding to the positive sample image is greater than the similarity between the image features corresponding to the astronomical image and the image features corresponding to the negative sample image; Determine the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, specifically including: Determine a first loss value according to the deviation between the classification result and the classification label, determine a second loss value according to the deviation between the regression result and the regression label, and determine a third loss value according to the similarities between the astronomical image and the positive sample image and the negative sample image respectively; Determine the total loss value according to the first loss value, the second loss value, and the third loss value, where the smaller the deviation between the classification result and the classification label, the smaller the first loss value, the smaller the deviation between the regression result and the regression label, the smaller the second loss value, the greater the similarity between the astronomical image and the positive sample image, the smaller the third loss value, and the smaller the similarity between the astronomical image and the negative sample image, the smaller the third loss value.

[0009] Optionally, determining a total loss value according to the first loss value, the second loss value, and the third loss value specifically includes: Weighting the first loss value, the second loss value, and the third loss value according to a first weight corresponding to the first loss value, a second weight corresponding to the second loss value, and a third weight corresponding to the third loss value to obtain a total loss value.

[0010] Optionally, the method further includes: Adjusting the first weight, the second weight, and / or the third weight according to a loss value difference, where the loss value difference includes: a difference between the first loss value and the second loss value, a difference between the first loss value and the third loss value, and a difference between the second loss value and the third loss value.

[0011] An embodiment of the present application provides a retrieval method based on astronomical images, including: Receiving a retrieval request; Matching the image features carried in the retrieval request with the image features corresponding to each astronomical image stored in a preset database to match target features, where the image features corresponding to each astronomical image included in the preset database are obtained through the feature extraction layer trained by the above model training method; Returning the astronomical image corresponding to the target feature as a retrieval result.

[0012] An embodiment of the present application provides a model training device, including: An acquisition module, configured to acquire astronomical images; A determination module, configured to determine a regression label and a classification label corresponding to the astronomical image, where the regression label is used to represent the features of the celestial body or galaxy corresponding to the astronomical image; An input module, configured to input the astronomical image into a classification model to be trained, so as to determine the image features corresponding to the astronomical image through the feature extraction layer in the classification model, input the image features into the classification layer in the classification model to obtain a classification result corresponding to the astronomical image, and input the image features into the regression layer in the classification model to obtain a regression result corresponding to the astronomical image; A training module, configured to determine a total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, so as to train at least the feature extraction layer in the classification model according to the total loss value.

[0013] An embodiment of the present application provides a retrieval device based on astronomical images, including: A receiving module, configured to receive a retrieval request; A matching module, configured to match the image features to be matched carried in the retrieval request with the image features corresponding to each astronomical image stored in a preset database, so as to match target features, where the image features corresponding to each astronomical image included in the preset database are obtained through the feature extraction layer trained by the above-mentioned model training method; A returning module, configured to return the astronomical image corresponding to the target features as a retrieval result.

[0014] An embodiment of the present application provides a computer-readable storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned model training method or the retrieval method based on astronomical images is implemented.

[0015] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: The model training method and the retrieval method based on astronomical images provided in the embodiments of the present application, after obtaining an astronomical image, determine the regression label and the classification label corresponding to the astronomical image. Then, input the astronomical image into a classification model to be trained, so as to determine the image features corresponding to the astronomical image through the feature extraction layer in the classification model, and input the astronomical image into the classification layer in the classification model to obtain the classification result corresponding to the astronomical image, and input the image features into the regression layer in the classification model to obtain the regression result corresponding to the astronomical image. According to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, determine the total loss value corresponding to the astronomical image, so as to train at least the feature extraction layer in the classification model according to the total loss value. Then, according to the trained feature extraction layer, the image features corresponding to each astronomical image can be extracted, and the astronomical image and its corresponding image features can be stored correspondingly. Furthermore, in the subsequent astronomical image retrieval process, the required astronomical image can be retrieved by means of image feature matching.

[0016] As can be seen from the above method, by training the classification model from two perspectives of enabling the classification model to obtain accurate classification results and accurate celestial body features, the feature extraction layer in the trained classification model can accurately extract the image features of astronomical images. Furthermore, in the subsequent retrieval process, the required astronomical image can be accurately matched through feature matching between astronomical image features, which provides great convenience for astronomical research. Description of the Drawings

[0017] The drawings described herein are used to provide a further understanding of the present specification, and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present specification and do not constitute an improper limitation to the present application. In the drawings: Figure 1 Schematic flowchart of a model training method provided by an embodiment of the present application; Figure 2 Schematic diagram of training a classification model with multiple heads provided by an embodiment of the present application; Figure 3 Schematic flowchart of a retrieval method based on astronomical images provided by an embodiment of the present application; Figure 4 Schematic diagram of a model training device provided by an embodiment of the present application; Figure 5 Schematic diagram of a retrieval device based on astronomical images provided by an embodiment of the present application; Figure 6 For an embodiment of the present application corresponding to Figure 1 or Figure 3 Schematic structural diagram of an electronic device. Detailed implementation manners

[0018] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this specification without creative efforts belong to the scope of protection of the present application.

[0019] The following will detail the technical solutions provided by each embodiment of the present application with reference to the drawings.

[0020] Figure 1 Schematic flowchart of a model training method provided by an embodiment of the present application, including the following steps: S101: Obtain astronomical images.

[0021] In the embodiment of the present application, astronomical images required for training the classification model can be obtained first. Among them, the obtained astronomical images can come from some existing image sets in the astronomical field.

[0022] The execution subject of the model training method in the embodiment of the present application can be a terminal device such as a desktop computer or a laptop computer, or a client installed in the terminal device, or a server, or a dedicated training machine specifically used for model training, etc. For the sake of description, the model training method provided by the embodiment of the present application will be described below only by taking the server as an example.

[0023] S102: Determine the regression label and classification label corresponding to the astronomical image, where the regression label is used to represent the characteristics of the celestial body or galaxy corresponding to the astronomical image.

[0024] After obtaining the above astronomical images, the terminal device needs to further determine the regression label and classification label corresponding to the astronomical images. Among them, the classification label is used to indicate which category the celestial body or galaxy corresponding to the astronomical image belongs to. Here, the category mainly reflects the structural characteristics of the celestial body or galaxy corresponding to the astronomical image. For example, whether the shape of the galaxy is elliptical, how many arms the galaxy has, whether the galaxy is synthetic or distributed separately, and so on.

[0025] The above classification labels can be pre-annotated in the image set in the astronomical field. Therefore, when the terminal device obtains these astronomical images from the above image set, it can also determine the classification labels corresponding to these astronomical images at the same time.

[0026] The above regression label is used to represent the characteristics of the celestial body or galaxy corresponding to the astronomical image. This characteristic more reflects the characteristics of the celestial body or galaxy in the observation phenomenon, such as redshift, magnitude, ellipticity, brightness, etc. And these regression labels can be obtained by the terminal device querying from the preset astronomical data according to the coordinates of the celestial body or galaxy corresponding to the astronomical image.

[0027] That is, for any astronomical image, the terminal device can first obtain the coordinates of the celestial body or galaxy corresponding to the astronomical image. The coordinates can be recorded in the above image set. Therefore, when the terminal device obtains the astronomical image from the above image set, it can also determine the coordinates of the celestial body or galaxy corresponding to the astronomical image at the same time.

[0028] After that, the terminal device can generate a retrieval instruction according to the coordinates, so as to find the astronomical data matching the coordinates from the preset astronomical data through the retrieval instruction, and then determine the regression label of the celestial body or galaxy corresponding to the astronomical image from the retrieved astronomical data. Among them, after the terminal device obtains these astronomical data, it can identify the required regression label from these astronomical data by means such as character recognition, semantic analysis, and audio recognition (if the astronomical data contains audio files).

[0029] In addition, for some specific regression labels (such as the brightness, color, etc. of the celestial body or galaxy corresponding to the astronomical image), the terminal device can also determine them by means of image processing. Specifically, the astronomical image obtained by the terminal device can be an RGB image with color. Then, the terminal device can obtain the color information of the astronomical image by means of image channel splitting. For the brightness of the celestial body or galaxy, the terminal device can first convert the obtained astronomical image into a grayscale image, and then judge whether the center point of the astronomical image is within the galaxy contour. If so, the terminal device can determine the brightness of the celestial body or galaxy corresponding to the astronomical image according to the pixel grayscale values within the galaxy contour in the astronomical image.

[0030] To further enhance the training effect of the classification model and enable it to more accurately represent the image features of astronomical images, before the terminal device inputs the astronomical image into the classification model to be trained, it can first perform image preprocessing on the astronomical image using a preset preprocessing method to obtain the preprocessed image.

[0031] The preprocessing methods here can include: adjusting the image size, image enhancement, adding noise, image rotation, etc. Among them, after the terminal device obtains the astronomical image, it can uniformly adjust it to a fixed size (such as 224. Choosing to adjust the astronomical image to 224 mainly considers the proportion of the celestial body or galaxy in the image. If the image is too small, the celestial body features that the subsequent classification model can learn are insufficient, resulting in poor final training effects). Then, Gaussian noise can be randomly added to the astronomical image adjusted to a fixed size, and then, through coarseDropout, some areas of the astronomical image can be set as black areas, and the processed image can be rotated slightly.

[0032] Adding Gaussian noise to the astronomical image can further improve the feature expression ability of the classification model for astronomical images containing noise, enabling users to obtain more accurate retrieval results based on the image features determined by the classification model during subsequent retrieval processes. Selecting coarseDropout to set some areas of the astronomical image as black areas is actually also to improve the feature expression ability of the classification model for astronomical images. The above-mentioned slight rotation of the astronomical image belongs to the method of image enhancement, and the ultimate goal is also to further improve the feature expression ability of the classification model for astronomical images.

[0033] Of course, the above preprocessing methods can also include methods such as modifying the color of the astronomical image and cropping parts of it. Generally speaking, through the preprocessing method, the image features of the original astronomical image can be changed to a certain extent. Then, through the model training method of this application, the classification model can still accurately determine the image features of the astronomical image even under certain degrees of interference. From another perspective, after preprocessing the astronomical image through the above preprocessing method, the classification model can learn how to extract the key features that can represent the characteristics of the corresponding astronomy of the astronomical image during the training process, making the classification model more accurate in expressing the image features of the astronomical image.

[0034] It should be noted that the celestial bodies or galaxies corresponding to the astronomical images in the embodiments of this application can include, for example, stars, planets, dwarf planets, satellites, asteroids, comets, star clusters, interstellar matter, etc.

[0035] S103: Input the astronomical image into the classification model to be trained, determine the image features corresponding to the astronomical image through the feature extraction layer in the classification model, input the image features into the classification layer in the classification model to obtain the classification result corresponding to the astronomical image, and input the image features into the regression layer in the classification model to obtain the regression result corresponding to the astronomical image.

[0036] In the embodiments of the present application, an existing model such as zoobot can be used as the classification model. However, the existing zoobot only has a classification function. Therefore, if the existing zoobot is trained, the feature expression ability of the classification model trained only from the classification perspective is not accurate, which will affect subsequent astronomical image retrieval.

[0037] Therefore, on the basis of the existing zoobot classification model, a multi-head mechanism can be introduced, so that on the basis of the classification perspective, the classification model can be trained by combining other perspectives additionally, enabling the trained classification model to have accurate feature expression.

[0038] In specific implementation, the backbone network of the zoobot model can be retained as the feature extraction layer, and then on this basis, a regression layer (regression head) is added, so that after the astronomical image is input into the classification model, the classification model can determine the image features corresponding to the astronomical image through the feature extraction layer, and then input the image features into the original classification layer and the added regression layer in the classification model respectively, so as to obtain the classification result and the regression result for the astronomical image respectively.

[0039] In the embodiments of the present application, the EfficientNet B0 can be selected as the feature extraction layer in the classification model. Its advantage is that it has a relatively large network scale, but the number of parameters in the B0 version is small, which is more suitable for large-scale real-time processing of astronomical data and has better feature extraction ability.

[0040] S104: Determine the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, and at least train the feature extraction layer in the classification model according to the total loss value.

[0041] After obtaining the above classification result and regression result, the terminal device can determine the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, and then at least train the feature extraction layer in the classification model according to the total loss value.

[0042] Among them, if the deviation between the classification result and the classification label is smaller, it indicates that the classification result of the classification model is more accurate, and the image features determined by the feature extraction layer in the classification model are also more accurate. Similarly, if the deviation between the regression result and the regression label is smaller, it indicates that the regression result of the classification model is more accurate, and the image features determined by the feature extraction layer in the classification model are also more accurate.

[0043] Therefore, in the embodiment of the present application, the terminal device can determine the first loss value according to the deviation between the classification result and the classification label, and determine the second loss value according to the deviation between the regression result and the regression label, so as to determine the total loss value according to the first loss value and the second loss value, and then train the classification model with the minimization of the total loss value as a constraint condition.

[0044] In the above process, the smaller the deviation between the classification result and the classification label, the smaller the first loss value, and the smaller the deviation between the regression result and the regression label, the smaller the second loss value.

[0045] As can be seen from the above method, by training the classification model from two perspectives of enabling the classification model to obtain accurate classification results and accurate celestial body features, the feature extraction layer in the trained classification model can accurately extract the image features of astronomical images. Then, in the subsequent retrieval process, the required astronomical images can be accurately matched through feature matching between astronomical image features, which provides great convenience for astronomical research.

[0046] In order to further improve the feature expression ability of the feature extraction layer in the classification model, metric learning can be further introduced in the embodiment of the present application, so as to add the loss value obtained by metric learning to the total loss value to train the classification model.

[0047] Specifically, the terminal device can input multiple astronomical images into the classification model to be trained. In this process, the classification model can respectively determine the image features corresponding to each astronomical image through its internal feature extraction layer.

[0048] After that, for each astronomical image, the similarity between the image features corresponding to the astronomical image and the image features corresponding to other astronomical images can be determined through the image features between each astronomical image, and the positive sample image and the negative sample image corresponding to the astronomical image can be determined according to the determined similarity. Among them, the similarity between the image features corresponding to the astronomical image and the positive sample image is greater than the similarity between the image features corresponding to the astronomical image and the negative sample image.

[0049] In the above process, there may be a variety of ways to determine positive and negative sample images. For example, the terminal device may determine other astronomical images whose similarity is not less than a preset similarity threshold as positive sample images corresponding to the astronomical image, and determine other astronomical images whose similarity is less than the preset similarity threshold as negative sample images corresponding to the astronomical image. For another example, after determining the similarity between an astronomical image and other astronomical images, the terminal device may use other astronomical images with the highest similarity as positive sample images corresponding to the astronomical image, and use the remaining other astronomical images as negative sample images corresponding to the astronomical image. Other examples are not given here one by one.

[0050] The similarity between the corresponding image features of the above-mentioned astronomical images can be calculated using methods such as Euclidean distance, cosine similarity, etc., and the embodiments of the present application do not make specific limitations on this.

[0051] Furthermore, the terminal device may determine a third loss value according to the similarity between the astronomical image and the positive sample image and the negative sample image, respectively. The purpose of introducing metric learning is to widen the similarity gap between the astronomical image and the positive sample image and the negative sample image, that is, to increase the similarity between the astronomical image and its corresponding positive sample image as much as possible, and to reduce the similarity between the astronomical image and its corresponding negative sample image.

[0052] Therefore, if the similarity between the astronomical image and the positive sample image is greater, the third loss value is smaller, and if the similarity between the astronomical image and the negative sample image is smaller, the third loss value is smaller.

[0053] The terminal device can determine the total loss value based on the first loss value, the second loss value and the third loss value, and then train the above classification model by minimizing the total loss value.

[0054] In addition, in the embodiment of the present application, each loss value also corresponds to a corresponding weight. Therefore, in the process of determining the above-mentioned total loss value, the terminal device can weight the first loss value, the second loss value, and the third loss value according to the first weight corresponding to the first loss value, the second weight corresponding to the second loss value, and the third weight corresponding to the third loss value to obtain the total loss value. For details, refer to the following formula (1): (1) in, is the first loss value, is the first weight corresponding to the first loss value, is the second loss value, is the second weight corresponding to the second loss value, is the third loss value, is the third weight corresponding to the third loss value, is the total loss value determined by weighted summation.

[0055] During the above model training process, if a fixed weight method is adopted, it may cause the classification model to overly focus on the results in a certain aspect during training, resulting in the distortion of the feature expression in the feature extraction layer of the classification model. For example, when the first weight, the second weight, and the third weight are all fixed values, if the first loss value determined based on the classification result is large, then during the training of the classification model, the classification model pays more attention to the learning of classification knowledge, resulting in the feature extraction layer in the classification model being more inclined to reflect the classification features of astronomical images in terms of feature expression, while reducing or even ignoring the features of astronomical images in other aspects.

[0056] To avoid the occurrence of the above situation, the terminal device can adjust the weights corresponding to different loss values during the above model training process, so that the classification model can learn various aspects of knowledge evenly and ensure that the feature extraction layer can comprehensively consider the characteristics of astronomical images in terms of feature expression.

[0057] Specifically, the terminal device can adjust the first weight, the second weight, and / or the third weight according to the loss value difference. Here, the loss value difference mentioned includes: the difference between the first loss value and the second loss value, the difference between the first loss value and the third loss value, and the difference between the second loss value and the third loss value. Specifically, refer to the following formula (2): (2) In the above formula, can be understood as the first weight to be adjusted, and the specific adjustment , is the second weight to be adjusted, and the specific adjustment , is the third weight to be adjusted, and the specific adjustment .

[0058] During the process of adjusting the weights, if the loss value difference between two loss values is large, the difference between the weights corresponding to these two loss values can be adjusted in the opposite direction. For example, if the first loss value is significantly greater than the second loss value, the value of the first weight can be reduced, and at the same time, the value of the second weight can be increased; for another example, if the first loss value is significantly less than the third loss value, the value of the first weight can be increased while keeping the value of the third weight unchanged. Other situations will not be exemplified one by one here.

[0059] As can be seen from the above process, the terminal device adjusts each weight according to the loss value difference, which can ensure that the contribution degree of each loss value and its corresponding weight to the total loss value is close, and the situation that the classification model overly focuses on a certain aspect of knowledge will not occur, thereby further ensuring the accurate feature expression of the feature extraction layer in the classification model.

[0060] In addition, as can be seen from the above formula (2), it also has a certain inhibitory effect on the reverse adjustment of the weights, preventing the situation that excessive adjustment affects the training effect. That is, assuming that it is necessary to increase the first weight, then it is necessary to decrease the value of, and once the value of is decreased, then in and remaining unchanged, the value as a whole decreases.

[0061] After training the classification model in the above manner, the trained classification model can be verified through the validation set. When it is determined that the accuracy of the trained classification model in processing the validation set is not lower than the preset accuracy threshold, it can be determined that the trained classification model passes the verification. Furthermore, the image features corresponding to each astronomical image can be determined through the trained classification model, and each astronomical image and the image features corresponding to each astronomical image are stored in a preset database in a corresponding manner.

[0062] If the trained classification model fails to pass the verification of the validation set, the network structure or parameters in the trained classification model can be adjusted, and the adjusted classification model can be trained through the training set until a classification model that meets the requirements is obtained.

[0063] Since the embodiments of the present application are mainly used to obtain the image features of accurate astronomical images, after obtaining the trained classification model in the above manner, only the internal feature extraction layer can be used to determine the image features of each astronomical image in the subsequent process. Of course, the above-trained classification model can also be used as a whole, and can be used to determine the classification results and regression results of some newly observed astronomical images in the subsequent process.

[0064] To further describe the model training method provided by the embodiments of the present application, it will be described below in conjunction with the structure of the classification model, as Figure 2 shown.

[0065] Figure 2 is a schematic diagram of training a classification model with multiple heads provided by the embodiments of the present application.

[0066] In Figure 2In the classification model shown, the Encoder is the feature extraction layer in the classification model. In this classification model, there are also a classification head, a regression head, and an additional head. Among them, the classification head is mainly used to obtain the classification result of the astronomical image based on the image features of the astronomical image output by the Encoder, and the regression head is used to obtain the regression result of the astronomical image based on the image features of the astronomical image output by the Encoder. And in Figure 2 the additional head shown can determine the brightness of the astronomical image according to the image features of the astronomical image output by the Encoder.

[0067] After that, the classification model can be trained by combining metric learning and the determined classification labels and regression labels.

[0068] It should be noted that in the above classification model, the additional head can be set according to actual needs. That is, if it is necessary to make the feature extraction layer in the classification model additionally refer to the characteristics of celestial bodies or galaxies in other aspects in terms of feature expression, a corresponding head can be added to the classification model to further improve the feature expression ability of the feature extraction layer.

[0069] After obtaining the image features corresponding to each astronomical image in the above manner, in the subsequent process, the required astronomical images can be retrieved by means of feature matching. The retrieval method based on astronomical images will be continued to be introduced below.

[0070] Figure 3 FIG. is a schematic flow chart of a retrieval method based on astronomical images provided by an embodiment of the present application, including the following steps: S301: Receive a retrieval request.

[0071] In the embodiment of the present application, the execution subject required to execute the retrieval method based on astronomical images can be a terminal device such as a desktop computer or a laptop computer, or a client installed in the terminal device, or a server. For the sake of description, below, only the server will be taken as an example to illustrate the retrieval method based on astronomical images provided by the embodiment of the present application.

[0072] When the user searches for astronomical images according to actual needs, a retrieval operation can be performed. The terminal device or client used by the user can generate a retrieval request according to the retrieval operation performed by the user, and send the retrieval request to the server, and the server executes the retrieval task corresponding to the retrieval request.

[0073] Among them, the user can input the image features of the astronomical image in the terminal device or the client, and the terminal device or the client can generate a retrieval request based on the image features to retrieve astronomical images with similar features to the image features in the subsequent process.

[0074] The image features input by the user can be generated by the classification model trained by the above method or the feature extraction layer in the classification model. That is, the user inputs the current astronomical image into the feature extraction layer in the classification model to obtain the image features corresponding to the astronomical image.

[0075] S302: Match the image features to be matched carried in the retrieval request with the image features corresponding to each astronomical image saved in the preset database to match the target features. The image features corresponding to each astronomical image included in the preset database are obtained through the feature extraction layer trained by the above model training method.

[0076] The server can match the image features to be matched carried in the above retrieval request with the image features corresponding to the astronomical images saved in the preset database, and determine the target features that highly match the image features to be matched by calculating the similarity between the image features. Among them, there may be multiple target features mentioned here. That is, the server can determine the image features with a similarity higher than the set threshold with the image features to be matched as the matched target features.

[0077] S303: Return the astronomical image corresponding to the target feature as the retrieval result.

[0078] The server can further determine the astronomical image corresponding to the target feature in the above preset database as the retrieval result and return it to the user, so that the user can conduct astronomical research based on the returned astronomical image.

[0079] The above is a model training method and a retrieval method based on astronomical images provided by one or more embodiments of the present application. Based on the same idea, the embodiments of the present application also provide a corresponding model training device and a retrieval device based on astronomical images, as Figure 4 、 Figure 5 shown.

[0080] Figure 4 The following is a schematic diagram of a model training device provided by an embodiment of the present application, specifically including: An acquisition module 401, configured to acquire astronomical images; A determination module 402, configured to determine the regression label and classification label corresponding to the astronomical image. The regression label is used to represent the features of the celestial body or galaxy corresponding to the astronomical image; An input module 403 is configured to input the astronomical image into a classification model to be trained, determine image features corresponding to the astronomical image through a feature extraction layer in the classification model, input the image features into a classification layer in the classification model to obtain a classification result corresponding to the astronomical image, and input the image features into a regression layer in the classification model to obtain a regression result corresponding to the astronomical image. A training module 404 is configured to determine a total loss value corresponding to the astronomical image according to a deviation between the classification result and the classification label and a deviation between the regression result and the regression label, and at least train the feature extraction layer in the classification model according to the total loss value.

[0081] Optionally, the regression label includes the brightness of the celestial body corresponding to the celestial image. The determining module 402 is specifically configured to determine whether the center point of the astronomical image is within the galaxy contour; if so, determine the brightness of the celestial body corresponding to the astronomical image according to the pixel gray values within the galaxy contour in the astronomical image.

[0082] Optionally, the apparatus further includes: A preprocessing module 405 is configured to perform image preprocessing on the celestial image by using a preset preprocessing method to obtain a preprocessed image before the input module 403 inputs the astronomical image into the classification model to be trained, where the preprocessing method includes at least one of adjusting the image size, image enhancement, adding noise, and image rotation. The input module 403 is specifically configured to input the preprocessed image into the classification model to be trained.

[0083] Optionally, before the training module 404 determines the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label and the deviation between the regression result and the regression label, the input module 403 is further configured to determine a similarity between the image features corresponding to the astronomical image and the image features corresponding to other astronomical images through the feature extraction layer; determine a positive sample image corresponding to the astronomical image and a negative sample image corresponding to the astronomical image according to the similarity, where the similarity between the image features corresponding to the astronomical image and the image features corresponding to the positive sample image is greater than the similarity between the image features corresponding to the astronomical image and the image features corresponding to the negative sample image. The training module 404 is specifically configured to determine a first loss value according to the deviation between the classification result and the classification label, determine a second loss value according to the deviation between the regression result and the regression label, and determine a third loss value according to the similarity between the astronomical image and the positive sample image and the negative sample image respectively; determine the total loss value according to the first loss value, the second loss value, and the third loss value, where the smaller the deviation between the classification result and the classification label, the smaller the first loss value, the smaller the deviation between the regression result and the regression label, the smaller the second loss value, the greater the similarity between the astronomical image and the positive sample image, the smaller the third loss value, and the smaller the similarity between the astronomical image and the negative sample image, the smaller the third loss value.

[0084] Optionally, the training module 404 is specifically configured to weight the first loss value, the second loss value, and the third loss value according to the first weight corresponding to the first loss value, the second weight corresponding to the second loss value, and the third weight corresponding to the third loss value, to obtain the total loss value.

[0085] Optionally, the training module 404 is further configured to adjust the first weight, the second weight, and / or the third weight according to the loss value difference, where the loss value difference includes: the difference between the first loss value and the second loss value, the difference between the first loss value and the third loss value, and the difference between the second loss value and the third loss value.

[0086] Figure 5 The figure is a schematic diagram of a retrieval device based on astronomical images provided by an embodiment of the present application, specifically including: A receiving module 501, configured to receive a retrieval request; A matching module 502, configured to match the image features carried in the retrieval request with the image features corresponding to each astronomical image stored in a preset database, to match out target features, where the image features corresponding to each astronomical image included in the preset database are obtained through the feature extraction layer trained by the above model training method; A returning module 503, configured to return the astronomical image corresponding to the target features as a retrieval result.

[0087] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the above Figure 1 provided model training method or the above Figure 3 provided retrieval method based on astronomical images.

[0088] This specification also providesFigure 6 A schematic structural diagram of an electronic device corresponding to Figure 1 or Figure 3 is shown. As Figure 6 shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above Figure 1 model training method or the above Figure 3 astronomical image-based retrieval method.

[0089] Of course, in addition to the software implementation method, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or logic devices.

[0090] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structures of diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many method flow improvements today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by a user's programming of the device. Designers can program themselves to "integrate" a digital system onto a single PLD, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, today, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compilers used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a hardware description language (HDL), and there is not just one type of HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that by simply performing some logical programming on the method flow using the above-mentioned several hardware description languages and programming it into an integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0091] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to make the controller implement the same function in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or the structures within the hardware component.

[0092] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0093] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0094] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0095] This specification is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or the functions specified in one or more of the blocks.

[0096] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or the functions specified in one or more of the blocks.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or the functions specified in one or more of the blocks.

[0098] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0099] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash RAM. The memory is an example of computer-readable media.

[0100] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0101] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0102] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system or a computer program product. Therefore, this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0104] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts in the method embodiments for relevant details.

[0105] The above description is only for the embodiments of this specification and is not intended to limit this specification. For those skilled in the art, various modifications and changes can be made to this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this specification.

Claims

1. A model training method, characterized in that, Including: Obtaining astronomical images; Determining a regression label and a classification label corresponding to the astronomical image, where the regression label is used to represent the characteristics of the celestial body or galaxy corresponding to the astronomical image; Inputting the astronomical image into a classification model to be trained, determining the image features corresponding to the astronomical image through the feature extraction layer in the classification model, inputting the image features into the classification layer in the classification model to obtain the classification result corresponding to the astronomical image, and inputting the image features into the regression layer in the classification model to obtain the regression result corresponding to the astronomical image; Determining the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, and training at least the feature extraction layer in the classification model according to the total loss value.

2. The method according to claim 1, wherein The regression label includes the brightness of the celestial body corresponding to the celestial image; Determining the regression label corresponding to the astronomical image specifically includes: Judging whether the center point of the astronomical image is within the galaxy contour; If so, determining the brightness of the celestial body corresponding to the astronomical image according to the pixel gray values within the galaxy contour in the astronomical image.

3. The method according to claim 1, wherein Before inputting the astronomical image into the classification model to be trained, the method further includes: Performing image preprocessing on the celestial image by using a preset preprocessing method to obtain a preprocessed image, where the preprocessing method includes at least one of adjusting the image size, image enhancement, adding noise, and image rotation; Inputting the astronomical image into the classification model to be trained specifically includes: Inputting the preprocessed image into the classification model to be trained.

4. The method according to claim 1, wherein Before determining the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, the method further includes: Determining the similarity between the image features corresponding to the astronomical image and the image features corresponding to other astronomical images through the feature extraction layer; Determining the positive sample image corresponding to the astronomical image and the negative sample image corresponding to the astronomical image according to the similarity, where the similarity between the image features corresponding to the astronomical image and the image features corresponding to the positive sample image is greater than the similarity between the image features corresponding to the astronomical image and the image features corresponding to the negative sample image; Determining the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label specifically includes: Determining a first loss value according to the deviation between the classification result and the classification label, determining a second loss value according to the deviation between the regression result and the regression label, and determining a third loss value according to the similarity between the astronomical image and the positive sample image and the negative sample image respectively. Determine a total loss value according to the first loss value, the second loss value, and the third loss value, where the smaller the deviation between the classification result and the classification label, the smaller the first loss value, the smaller the deviation between the regression result and the regression label, the smaller the second loss value, the greater the similarity between the astronomical image and the positive sample image, the smaller the third loss value, and the smaller the similarity between the astronomical image and the negative sample image, the smaller the third loss value.

5. The method according to claim 4, wherein Determine a total loss value according to the first loss value, the second loss value, and the third loss value, specifically including: Weight the first loss value, the second loss value, and the third loss value according to the first weight corresponding to the first loss value, the second weight corresponding to the second loss value, and the third weight corresponding to the third loss value to obtain a total loss value.

6. The method according to claim 5, wherein The method further includes: Adjust the first weight, the second weight, and / or the third weight according to the loss value difference, where the loss value difference includes: the difference between the first loss value and the second loss value, the difference between the first loss value and the third loss value, and the difference between the second loss value and the third loss value.

7. A retrieval method based on astronomical images, characterized in that, Including: Receive a retrieval request; Match the image features carried in the retrieval request with the image features corresponding to each astronomical image stored in a preset database to match target features, where the image features corresponding to each astronomical image included in the preset database are obtained through a feature extraction layer trained by the model training method according to any one of claims 1 to 6; Return the astronomical image corresponding to the target features as a retrieval result.

8. A model training device, characterized in that, Including: An acquisition module for acquiring astronomical images; A determination module for determining the regression label and the classification label corresponding to the astronomical image, where the regression label is used to represent the features of the celestial body or galaxy corresponding to the astronomical image; An input module for inputting the astronomical image into a classification model to be trained, to determine the image features corresponding to the astronomical image through the feature extraction layer in the classification model, input the image features into the classification layer in the classification model to obtain the classification result corresponding to the astronomical image, and input the image features into the regression layer in the classification model to obtain the regression result corresponding to the astronomical image; A training module for determining the total loss value corresponding to the astronomical image according to the deviation between the classification result and the classification label, and the deviation between the regression result and the regression label, and training at least the feature extraction layer in the classification model according to the total loss value.

9. A retrieval device based on astronomical images, characterized in that, Including: A receiving module for receiving a retrieval request; A matching module for matching the image features carried in the retrieval request with the image features corresponding to each astronomical image stored in a preset database to match target features, where the image features corresponding to each astronomical image included in the preset database are obtained through a feature extraction layer trained by the model training method according to any one of claims 1 to 6; A return module, configured to return the astronomical image corresponding to the target feature as a retrieval result.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above claims 1 to 6 is implemented.

Images