Model training method and device, target detection method and device, computer equipment and storage medium

Abstract

The invention relates to a model training method and device, a target detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a sample image corresponding to a target object; the sample image is obtained according to a main prompt word of the target object; the main prompt word is used for describing characteristics of the target object; inputting the sample image and the subject cue word into a to-be-trained initial detection model, and performing target detection processing on the sample image based on the subject cue word through the initial detection model to obtain prediction annotation information of the sample image; according to combined annotation information and same annotation information between the predicted annotation information and target annotation information of the sample image, performing iterative training on the initial detection model to obtain a trained target detection model; the target annotation information is obtained by carrying out annotation processing on a target object in the sample image. By adopting the method, the target detection effect of the target detection model obtained by training can be improved, and the automatic labeling efficiency of the sample image can also be improved.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a model training method, target detection method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Object detection technology aims to automatically identify the type and location of non-human objects in images or videos using algorithms. With the rapid development of object detection technology, it has been gradually applied in various fields such as security monitoring, autonomous driving, intelligent manufacturing, and human-computer interaction.

[0003] Traditional object detection models typically require training on large-scale labeled datasets to learn the features of objects in images or videos. Therefore, the quality and quantity of the labeled dataset directly impact the performance of the object detection model. However, manually labeling data is both time-consuming and expensive, and faces challenges in certain application scenarios, such as few-shot and zero-shot detection, where it is difficult to obtain sufficient labeled data. Summary of the Invention

[0004] Therefore, it is necessary to provide a model training method, object detection method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the annotation efficiency of sample images to address the above-mentioned technical problems.

[0005] Firstly, this application provides a model training method. The method includes:

[0006] A sample image corresponding to the target object is obtained; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0007] The sample image and the main prompt word are input into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0008] Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0009] Secondly, this application provides a target detection method. The method includes:

[0010] Acquire the image to be detected;

[0011] The image to be detected is input into the target detection model, and the target detection model performs target detection processing on the image to be detected to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in the first aspect.

[0012] Thirdly, this application also provides a model training apparatus. The apparatus includes:

[0013] An image generation module is used to acquire a sample image corresponding to a target object; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0014] The annotation prediction module is used to input the sample image and the main prompt word into the initial detection model to be trained, and to perform target detection processing on the sample image based on the main prompt word through the initial detection model to obtain the predicted annotation information of the sample image;

[0015] The model training module is used to iteratively train the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, so as to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0016] In one embodiment, the image generation module is further configured to obtain image generation values ​​corresponding to the main prompt word and the background prompt word of the target object; the background prompt word is used to describe the additional characteristics of the target object and / or the background information of its environment; the image generation value is used to control the generation of different images; and through a text-based image model, based on the image generation value, image generation processing is performed on the main prompt word and the background prompt word to obtain the sample image corresponding to the main prompt word and the background prompt word.

[0017] In one embodiment, the model training device further includes a text-based image module, used to encode the main prompt word and the background prompt word using the text-based image model to obtain the main semantic information corresponding to the main prompt word and the background semantic information corresponding to the background prompt word; and to perform image generation processing on the main semantic information and the background semantic information based on the image generation value using the image generation model in the text-based image model to obtain the sample image corresponding to the main prompt word and the background prompt word.

[0018] In one embodiment, the model training device further includes an image annotation module for acquiring an image annotation model constructed based on a large-scale object detection model; the image annotation model is larger than the initial detection model; the large-scale object detection model is used to detect target objects in the image that match the input subject prompt; through the image annotation model, based on the subject prompt, the sample image is subjected to target detection processing for the target object to obtain the target type label and target location information of the target object in the sample image; the target annotation information is obtained according to the target type label and the target location information.

[0019] In one embodiment, the model training device further includes a position processing module, configured to obtain the combined annotation information based on the union of the predicted position information in the predicted annotation information and the target position information in the target annotation information; and to obtain the identical annotation information based on the intersection of the predicted position information and the target position information.

[0020] In one embodiment, the model training module is further configured to determine the intersection-union ratio (IU) between the predicted annotation information and the target annotation information based on the combined annotation information and the identical annotation information; obtain a loss value between the predicted annotation information and the target annotation information based on the IU; and iteratively update the model parameters of the initial detection model based on the loss value until a preset training termination condition is met, thereby obtaining the target detection model.

[0021] Fourthly, this application also provides a target detection device. The device includes:

[0022] The image acquisition module is used to acquire the image to be detected;

[0023] The target detection module is used to input the image to be detected into the target detection model, and to perform target detection processing on the image to be detected through the target detection model to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in the first aspect.

[0024] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0025] A sample image corresponding to the target object is obtained; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0026] The sample image and the main prompt word are input into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0027] Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0028] Sixthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0029] Acquire the image to be detected;

[0030] The image to be detected is input into the target detection model, and the target detection model performs target detection processing on the image to be detected to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in the first aspect.

[0031] Seventhly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0032] A sample image corresponding to the target object is obtained; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0033] The sample image and the main prompt word are input into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0034] Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0035] Eighthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0036] Acquire the image to be detected;

[0037] The image to be detected is input into the target detection model, and the target detection model performs target detection processing on the image to be detected to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in the first aspect.

[0038] Ninthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0039] A sample image corresponding to the target object is obtained; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0040] The sample image and the main prompt word are input into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0041] Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0042] Tenthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0043] Acquire the image to be detected;

[0044] The image to be detected is input into the target detection model, and the target detection model performs target detection processing on the image to be detected to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in the first aspect.

[0045] The aforementioned model training method, object detection method, device, computer equipment, storage medium, and computer program product acquire sample images corresponding to the target objects. The sample images are obtained based on the main prompts for the target objects. These main prompts describe the characteristics of the target objects. The sample images and main prompts are input into an initial detection model to be trained. The initial detection model performs object detection processing on the sample images based on the main prompts, obtaining predicted annotation information for the sample images. Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample images, the initial detection model is iteratively trained to obtain a trained object detection model. Target annotation information is obtained by annotating the target objects in the sample images. This method first acquires rich sample images based on the main prompts for the target objects, improving the diversity of the sample images. Then, the initial detection model detects the predicted annotation information of the sample images. The initial detection model is trained using the predicted annotation information and the target annotation information of the sample images, which not only improves the object detection performance of the trained object detection model but also improves the efficiency of automatic annotation of the sample images. Attached Figure Description

[0046] Figure 1 This is a diagram illustrating the application environment of a model training method in one embodiment;

[0047] Figure 2 This is a flowchart illustrating a model training method in one embodiment;

[0048] Figure 3 This is a flowchart illustrating the steps of obtaining a sample image corresponding to a target object in one embodiment;

[0049] Figure 4 This is a schematic diagram illustrating the principle of calculating combined annotation information and identical annotation information in one embodiment.

[0050] Figure 5 This is a flowchart illustrating a target detection method in one embodiment;

[0051] Figure 6 This is a flowchart illustrating the model training method in another embodiment;

[0052] Figure 7 This is a schematic diagram of the process for generating training samples in one embodiment;

[0053] Figure 8 This is a flowchart illustrating the process of annotating sample images in one embodiment;

[0054] Figure 9 This is a flowchart illustrating the training process of a detection model in one embodiment;

[0055] Figure 10This is a structural block diagram of a model training device in one embodiment;

[0056] Figure 11 This is a structural block diagram of a target detection device in one embodiment;

[0057] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0060] The model training method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is a smart home system, which may include: a router, a gateway, terminals 101 (such as mobile phones, tablets, and laptops), a server 102, and multiple sub-devices (such as cameras, sensors, and switches). In this smart home system, sub-devices can connect to the gateway via ZIGBEE / Bluetooth / WiFi, and the gateway and user terminals can connect to the router via WiFi. Additionally, terminals can also establish network connections to the server via 2G / 3G / 4G / 5G, WiFi, etc., thereby obtaining data sent by the server.

[0061] The terminal is used to provide local services to users. The server can be implemented using a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0062] In an exemplary embodiment, an electronic device obtains a sample image corresponding to a target object based on the target object's main cue words; wherein, the main cue words are used to describe the characteristics of the target object; then, the electronic device inputs the sample image and the main cue words into an initial detection model to be trained; through the initial detection model, the sample image is processed for target detection based on the main cue words to obtain the predicted annotation information of the sample image; based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained, and finally the electronic device obtains a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0063] In one embodiment, such as Figure 2 As shown, a model training method is provided. Taking the application of this method to an electronic device as an example, the electronic device can be... Figure 1 In any of the following devices: server, terminal, or gateway. The above method includes the following steps:

[0064] Step S201: Obtain the sample image corresponding to the target object; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0065] Here, sample images refer to images generated based on the main prompts for the target object. Sample images can be used as training data to train the detection model.

[0066] The target object refers to the object to be detected. Target objects include, but are not limited to, vehicles, human bodies, and animals.

[0067] Here, the main prompt word refers to a section of text describing the main characteristics of the target object. These main characteristics include, but are not limited to, the object's color, shape, and name. For example, the main prompt word could be "a red car." The main prompt word can be user-inputted or automatically generated. Specifically, the electronic device can receive main prompt words input by the user for the target object, download open-source main prompt words from the internet, or use an open-source prompt word sample set to obtain the main prompt word for the target object. For each main prompt word, multiple (e.g., 200-500) different sample images can be generated using text-to-image models to increase the diversity of sample images used for model training.

[0068] Step S202: Input the sample image and the main prompt word into the initial detection model to be trained. Through the initial detection model, target detection processing is performed on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0069] The initial detection model refers to the model used to detect targets in input data (such as sample images) that match cue words (such as main cue words and background cue words). In practical applications, the initial detection model can be built using the Universal Detection Model (an open-source object detection model).

[0070] The prediction annotation information refers to the textual description of the target object's location and type in the sample image, obtained through model detection. The prediction annotation information includes the predicted type label of the target object and the predicted location information in text form. For example, the prediction annotation information could be "car: 100, 100, 200, 200", where "car" is the predicted type label representing the bounding box.

[0071] Specifically, the electronic device can construct an initial detection model to be trained based on large-scale object detection models such as YOLO-World (an open-source, open-vocabulary object detection model designed to address the limitations of traditional object detection models when facing new object categories) and Dino (Dual-Stage Implicit Object-Oriented Network, an open-source deep learning model for self-supervised visual learning). The electronic device inputs sample images and subject prompts into the initial detection model to be trained. The initial detection model automatically detects objects in the sample images based on the subject prompts. For example, the initial detection model can convert the subject prompts into semantic vectors, then match the visual features in the sample images with the semantic vectors (e.g., calculate the similarity between visual features and semantic vectors; a similarity higher than a threshold is considered a match). Finally, it outputs the predicted type label of the detected object with the highest matching probability, along with the predicted location information in text form (rather than bounding box annotations). The electronic device then sets the output predicted type label and predicted location information in text form as the predicted annotation information for the sample images.

[0072] Step S203: Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0073] In this context, combined annotation information refers to the union of two annotation information sets (such as predicted annotation information and target annotation information). Identical annotation information refers to the intersection of two annotation information sets.

[0074] Specifically, electronic devices can annotate targets in sample images using a larger-scale image annotation model, or manually annotate targets in sample images, ultimately obtaining target annotation information for the sample images and confirming this target annotation information as more reliable labels. The electronic device calculates combined annotation information based on the union of predicted and target annotation information for the sample images, and calculates identical annotation information based on the intersection of predicted and target annotation information for the sample images. Then, the electronic device uses the combined and identical annotation information to calculate the loss value of the initial detection model, and uses this loss value to iteratively update the model parameters of the initial detection model, resulting in a trained detection model. When a preset termination condition is met, the electronic device obtains the trained target detection model.

[0075] Furthermore, electronic devices can automatically label sample images using the trained target detection model, and can also continuously iterate and optimize the model using the labeled sample images. This not only enables the target detection model to continuously improve its performance, but also continuously improves the labeling efficiency and accuracy of sample images, thus realizing automatic iteration of the target detection model and automatic labeling of sample images.

[0076] In the above model training method, sample images corresponding to the target object are obtained; the sample images are obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object; the sample images and main prompt words are input into the initial detection model to be trained; the initial detection model performs target detection processing on the sample images based on the main prompt words to obtain the predicted annotation information of the sample images; the initial detection model is iteratively trained based on the combined annotation information and the same annotation information between the predicted annotation information and the target annotation information of the sample images to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample images. Using this method, rich sample images are first obtained based on the main prompt words of the target objects, improving the diversity of the sample images of the target objects; then, the predicted annotation information of the sample images is detected by the initial detection model, and the initial detection model is trained using the predicted annotation information and the target annotation information of the sample images. This not only improves the target detection effect of the trained target detection model, but also improves the efficiency of automatic annotation of sample images.

[0077] In one embodiment, such as Figure 3 As shown, step S201 above, obtaining the sample image corresponding to the target object, specifically includes the following:

[0078] Step S301: Obtain the image generation values ​​corresponding to the main prompt word and the background prompt word of the target object; the background prompt word is used to describe the additional characteristics of the target object and / or the background information of its environment; the image generation values ​​are used to control the generation of different images.

[0079] The image generation value refers to the parameter used to control the image generation process. In practical applications, this parameter controlling the image generation process can be a random seed; for example, the image generation value can be a number such as 1, 50, or 100.

[0080] Background information refers to a text describing the additional characteristics of the target object besides the main subject and / or its environment. Additional characteristics include, but are not limited to, the target object's environment, location, lighting, angle, obstructions, brand, and model. For example, a background information could be "A car of a certain brand and model in a well-lit yard; the camera captures the car at a height of approximately 3 meters, and the car is unobstructed." Background information can be user-inputted or automatically generated.

[0081] Specifically, to increase the diversity of the generated sample images, in addition to the main cue words, other descriptive cue words can be added, such as additional features of the target object and / or background information of the target object's environment. The electronic device can receive background cue words input by the user for the target object, download open-source background cue words from the internet, or automatically generate background cue words, thus obtaining the background cue words for the target object. Furthermore, each time a sample image is generated using the Wensheng image model, a random seed can be used as the image generation value. For example, this could be a random seed manually input by the user, or a random seed automatically generated by the electronic device according to a set parameter range.

[0082] Step S302: Using the text-generated image model, based on the image generation value, perform image generation processing on the main prompt word and the background prompt word to obtain sample images corresponding to the main prompt word and the background prompt word.

[0083] Text-to-image generation models refer to AI-generated models that automatically generate corresponding images based on user-input text descriptions (such as main and background prompts). For example, text-to-image generation models can be built based on Stable Diffusion (an open-source AI painting generation tool).

[0084] The image generation values ​​are parameter values ​​used to generate different images; for example, they could be values ​​used to initialize a random number generator. The main function of the image generation values ​​is to ensure the diversity and non-repetition of the generated images. The image generation values ​​determine the distribution of initial noise, thus indirectly affecting the texture, layout, color, and other details of the final image. When using the same cue words, model parameters, and image generation values, the model will generate completely identical images; however, changing the image generation values ​​(even if the cue words remain the same) will result in new images with different details and compositions.

[0085] Specifically, the generated image value, along with the subject and background cue words of the target object, are input into the text-based image model. The model then performs image generation processing on the subject and background cue words based on the generated image value, ultimately generating a sample image that matches both the subject and background cue words.

[0086] Image generation values ​​are used to further enhance the diversity of the generated sample images. It's important to note that different input image generation values ​​result in different generated sample images. For example, suppose the initial input topic prompt is "a red car," and the background prompt is "a car of a certain brand and model in a well-lit yard, the car's height as captured by the camera is approximately 3 meters, and the car is unobstructed." The image generation value is 256, and the text-generated image model generates sample image 1 based on this. Suppose the second input topic prompt is "a red car", and the background prompt is "a car of a certain brand and model in a well-lit yard, the height of the car captured by the camera is about 3m, and the car is unobstructed". The image generation value is 100 (different from the image generation value of 256 in sample image 1). Based on this, the text-generated image model generates sample image 2. Then, the generated sample image 1 and sample image 2 are different images. For example, both sample image 1 and sample image 2 contain a red car, and the position, ambient light, and angle of the car are the same. However, the image content details of sample image 1 are different from those of sample image 2. For example, the shape of the trees, the car sticker decorations, or the number of clouds in the sky in sample image 1 are different from those in sample image 2.

[0087] In this embodiment, by inputting the main prompt words describing the main body of the target object, the background prompt words describing the additional characteristics of the target object and / or the background information of its environment, and the image generation values ​​used to control the generation of different images into the text-based image model, rich information is provided for the text-based image model to generate sample images, which effectively improves the diversity of sample images and lays the foundation for subsequent model training and sample image annotation.

[0088] In one embodiment, step S302, which involves using a text-based graph model to generate images of the main prompt and background prompt based on image generation values, to obtain sample images corresponding to the main prompt and background prompt, specifically includes the following: encoding the main prompt and background prompt using a text-based graph model to obtain the main semantic information corresponding to the main prompt and the background semantic information corresponding to the background prompt; and using the image generation model in the text-based graph model to generate images of the main semantic information and background semantic information based on image generation values, to obtain sample images corresponding to the main prompt and background prompt.

[0089] In this context, the image generation model refers to the model used to generate images within the text-based image model. In practical applications, the image generation model is the main branch of the text-based image model and can be constructed using a diffusion model.

[0090] Specifically, using a text-based graph model based on Stable Diffusion, the main prompt and background prompt are encoded into semantic tokens (the smallest semantic unit after segmentation). The electronic device then obtains the semantic tokens corresponding to the main prompt (i.e., the main semantic information) and the semantic tokens corresponding to the background prompt (i.e., the background semantic information). The electronic device inputs these tokens into the image generation model within the text-based graph model. The image generation model generates corresponding images based on the main and background semantic information, ultimately outputting sample images that match the main and background prompts.

[0091] In this embodiment, based on the text-based image model, the semantic information of the subject and background prompts is accurately extracted by encoding them separately, ensuring that the semantics of key elements (such as the subject object and the scene) are accurately parsed and avoiding semantic information confusion. By fusing the subject semantic information and the background semantic information, a high-quality image with rich details and conforming to the subject and background prompts is generated, providing high-quality data for subsequent model training and sample image annotation.

[0092] In one embodiment, before iteratively training the initial detection model according to the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image in step S203 above, to obtain the trained target detection model, the method further includes: obtaining an image annotation model constructed based on the large-scale object detection model; the image annotation model is larger than the scale of the initial detection model; the large-scale object detection model is used to detect target objects in the image that match the input subject prompt words; through the image annotation model, based on the subject prompt words, target detection processing is performed on the sample image for the target objects to obtain the target type label and target location information of the target objects in the sample image; and target annotation information is obtained according to the target type label and target location information.

[0093] Among them, the "Everything Detection Big Model" refers to a large-scale visual perception model based on deep learning that can detect, locate, and recognize objects of any category in an image. The "Everything Detection Big Model" breaks through the category limitations of traditional object detection models (such as YOLO, which stands for "You Only Look Once," a deep learning-based object detection model) and achieves "open-world" general detection capabilities.

[0094] The location information includes the coordinates of the top-left vertex (x0, y0) of the bounding box of the target object, and the coordinates of the bottom-right vertex (x1, y1).

[0095] Specifically, the electronic device uses a large-scale object detection model, which is larger than the initial detection model, to construct an image annotation model, and then trains the image annotation model. The detection image and its subject prompt are input into the trained image annotation model. The trained image annotation model detects objects in the sample image that match the subject prompt (e.g., "red car"). For example, the trained image annotation model can extract multi-scale visual features from the detection image and perform text encoding on the subject prompt to obtain its semantic vector. The trained image annotation model can calculate the similarity (e.g., cosine similarity) between the visual features and semantic vectors of each region in the detection image, and regard regions with high similarity as the regions where the target object is located.

[0096] The trained image annotation model then outputs target type labels (e.g., "car") and location bounding boxes for objects in the sample image. Based on these bounding boxes, the target location information (e.g., "100, 100, 200, 200") in text form (rather than detection box annotations) is obtained for the object in the sample image. The electronic device sets the target type labels and text-based target location information as the target annotation information for the sample image. In this embodiment, the image annotation model based on the large-scale object detection model dynamically detects objects through main prompts, without needing to predefine fixed detection categories. This adapts to diverse annotation needs, and the output includes target type labels and text-based target location information. By setting the target type labels and target location information as the target annotation information for the sample image, a reliable data processing basis is provided for subsequent steps to train a small-scale initial detection model.

[0097] In one embodiment, before iteratively training the initial detection model to obtain the trained target detection model in step S203 based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the method further includes: obtaining combined annotation information based on the union of the predicted location information in the predicted annotation information and the target location information in the target annotation information; and obtaining identical annotation information based on the intersection of the predicted location information and the target location information.

[0098] Among them, the combined annotation information refers to the positional union between the predicted location information and the target location information.

[0099] Among them, the same labeled information refers to the location intersection between the predicted location information and the target location information.

[0100] Specifically, the electronic device calculates the position union between the predicted position information in the predicted annotation information and the target position information in the target annotation information, and sets the position union as the combined annotation information. The electronic device also calculates the position intersection between the predicted position information and the target position information, and sets the position intersection as the same annotation information.

[0101] For example, Figure 4 To illustrate the principle of calculating combined and identical annotation information, assume the target location information of the sample image output by the image annotation model is B1(100, 100, 200, 200), and the predicted location information of the sample image output by the initial detection model is B2(50, 100, 200, 150). Then, the combined annotation information (Union) can be represented as B1∪B2, as follows: Figure 4 The dark portion indicated by the Union in the middle represents the entire positions of B1 and B2; identical annotation information (Intersection) can be represented as B1∩B2, such as... Figure 4 The dark area shown on the left side of the Intersection chart represents the intersection of B1 and B2. Furthermore, the Intersection over Union (IoU) ratio between the target location information and the predicted location information can be expressed as the ratio between identical annotation information and combined annotation information, such as... Figure 4 The diagram on the right shows the Intersection over Union.

[0102] In this embodiment, the combined annotation information and identical annotation information of the sample image are calculated by the union and intersection of the predicted location information in the predicted annotation information and the target location information in the target annotation information, respectively. This provides a reliable data processing basis for determining the loss value of the initial detection model and updating the model parameters of the initial detection model using the loss value.

[0103] In one embodiment, step S203 above, which iteratively trains the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, to obtain a trained target detection model, specifically includes the following: determining the intersection-union ratio (IUR) between the predicted annotation information and the target annotation information for the combined annotation information and identical annotation information; obtaining the loss value between the predicted annotation information and the target annotation information based on the IUR; and iteratively updating the model parameters of the initial detection model based on the loss value until a preset training termination condition is met, thereby obtaining the target detection model.

[0104] Specifically, the electronic device can use combined annotation information and identical annotation information to calculate the intersection-union ratio (IUR) between the predicted location information and the target location information. The electronic device determines a loss value based on the IUR to measure the positional deviation between the predicted location information detected by the small-scale initial detection model and the target location information annotated by the large-scale image annotation model. Then, the model parameters of the initial detection model are iteratively updated using the loss value until a preset training termination condition is met, at which point training stops, and the electronic device obtains the target detection model.

[0105] In practical applications, such as Figure 4 As shown, based on the combined annotation information (B1∪B2) and the same annotation information (B1∩B2), the intersection-over-union (IoU) ratio between the target location information output by the image annotation model and the predicted location information output by the initial detection model can be obtained. The IoU ratio can be expressed by the following formula:

[0106]

[0107] After calculating the intersection-union ratio (IUR) between the target location information and the predicted location information, the electronic device further determines the loss value between the output of the image annotation model (i.e., the target location information) and the output of the initial detection model (i.e., the predicted location information) based on the IUR. Specifically, the IUR value can be directly used as the loss value, or the loss value can be further calculated based on the IUR using a loss function.

[0108] In this embodiment, by continuously reasoning and backpropagating the loss value of the initial detection model, the loss value of the initial detection model is continuously reduced. This allows the gap between the target location information annotated by the large-scale image annotation model and the predicted location information detected by the small-scale initial detection model to be continuously narrowed, thereby continuously improving the model performance of the trained initial detection model and greatly improving the accuracy of the trained target detection model in target detection.

[0109] In one embodiment, such as Figure 5 As shown, a model training method is provided. This embodiment illustrates the method by applying it to an electronic device. It is understood that this method can also be applied to a terminal, a server, or a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0110] Step S501: Obtain the image to be detected.

[0111] The image to be detected refers to the image that needs to undergo target detection processing.

[0112] Specifically, the device can receive images to be detected captured by a camera, obtain images to be detected from storage media, generate images to be detected using a text-based image model, and receive images to be detected uploaded by users, etc.

[0113] Step S502: Input the image to be detected into the target detection model, and use the target detection model to perform target detection processing on the image to be detected to obtain the target detection result of the target object in the image to be detected; the target detection model is trained by the above model training method.

[0114] Specifically, deploying a pre-trained, small-scale object detection model on the device effectively reduces the storage space required. The device inputs the acquired image to be detected into the object detection model, which automatically performs object detection processing on the image, yielding the object detection result. It should be noted that during the inference phase of the object detection model, either the main subject clues of the image to be detected are not required, or they can be input to assist in object detection.

[0115] By continuously generating sample images, automatically labeling sample images, and automatically iteratively training the object detection model, the precision and recall of the object detection model can be continuously improved, allowing it to be deployed in electronic devices such as camera equipment (e.g., webcam products). In practical applications, this can be achieved by generating vehicle-related sample images, automatically labeling vehicles in the sample images, training the object detection model's vehicle detection performance, and then deploying the object detection model on camera equipment. In vehicle recognition scenarios, camera equipment equipped with an object detection model can significantly improve the accuracy of vehicle recognition.

[0116] The training process for the object detection model is described in the model training method above and will not be repeated here.

[0117] In the above model training method, the pre-trained target detection model deployed on the device automatically performs target detection processing on the image to be detected, and obtains the target detection result of the image to be detected. This significantly improves the efficiency and accuracy of automatic image annotation, and also provides rich and high-quality training sample images for subsequent large-scale and diverse dataset training of the model.

[0118] In one embodiment, such as Figure 6 As shown, another model training method is provided. Taking the application of this method to an electronic device as an example, it includes the following steps:

[0119] Step S601: Obtain the image generation values ​​corresponding to the main prompt word and the background prompt word of the target object; the background prompt word is used to describe the additional characteristics of the target object and / or the background information of its environment; the main prompt word is used to describe the characteristics of the target object; the image generation value is used to control the generation of different images.

[0120] Step S602: Using the text-generated image model, based on the image generation value, perform image generation processing on the main prompt word and the background prompt word to obtain sample images corresponding to the main prompt word and the background prompt word.

[0121] Step S603: Input the sample image and the main prompt word into the initial detection model to be trained. Through the initial detection model, target detection processing is performed on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0122] Step S604: Obtain the image annotation model constructed based on the large-scale object detection model; the size of the image annotation model is larger than the size of the initial detection model; the large-scale object detection model is used to detect target objects in the image that match the input subject prompt word.

[0123] Step S605: Using an image annotation model, based on the main prompt words, target detection processing is performed on the sample image to obtain the target type label and target location information of the target in the sample image.

[0124] Step S606: Obtain target annotation information based on target type label and target location information.

[0125] Step S607: Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0126] The above model training method can achieve the following beneficial effects: First, it obtains rich sample images based on the main prompt words of the target object, thereby improving the diversity of the sample images of the target object; then, it detects the predicted annotation information of the sample images through the initial detection model, and uses the predicted annotation information and the target annotation information of the sample images to train the initial detection model, which not only improves the target detection effect of the trained target detection model, but also improves the efficiency of automatic annotation of sample images.

[0127] To more clearly illustrate the model training method provided in this disclosure, a specific embodiment is given below to describe the above-mentioned model training method in detail. Another model training method is also provided, which can be applied to electronic devices, and specifically includes the following:

[0128] (1) Generate training samples

[0129] Figure 7 This is a flowchart illustrating the process of generating training samples. Figure 7 As shown, electronic devices can construct text-based image models based on diffusion models such as Stable-Diffusion. Different prompts can be input into the text-based image model to increase the diversity of generated images. Random seeds (i.e., image generation values) can also be input into the text-based image model to generate multiple different images for each prompt. All generated images are saved and set as sample images. For example, each prompt can generate approximately 200-500 images. Each generation requires the user to input a random seed. The number of random seeds is the same as the number of generated images, and each random seed has a different value. Therefore, the 200-500 images correspond to different random seeds, resulting in different generated images.

[0130] Taking the generation of sample images of vehicles as an example, the main target vehicle is input into the text prompts of the text-based image generation model. The text-based image generation model encodes the text prompts into tokens, and then inputs these tokens into the image generation model (such as a diffusion model) within the text-based image generation model. The image generation model then generates the corresponding sample images of the vehicle based on the tokens. Each sample image requires the input of a separate text prompt. To increase the diversity of generated targets, other prompts can be added, such as location, lighting, angle, occlusion, vehicle brand, and vehicle type. For example: a small car in a well-lit yard with no obstructions, at a camera height of approximately 3 meters.

[0131] (2) Labeling sample images

[0132] Figure 8 This is a flowchart illustrating the process of labeling sample images. (For example...) Figure 8 As shown, the electronic device reads all the sample images saved in step (1) above. Target detection processing is performed on the target objects in the sample images using a large-scale image annotation model, outputting the target type labels and location bounding boxes of the target objects in the sample images. Based on the location bounding boxes, the target location information of the target objects in the sample images is obtained. The image annotation model can be constructed based on large-scale object detection models such as YOLO-World and DINO. The target type labels and target location information are saved as annotation files, i.e., target annotation information, which corresponds one-to-one with the sample images.

[0133] For example, an image annotation model built on a large-scale object detection model can use the input keyword "vehicle" as the detection target and continuously detect vehicle sample images generated by the text-based image model. It detects one image at a time, iterating until all vehicle sample images have been detected. The detected target location information is saved as an annotation file, which corresponds one-to-one with the sample images. For example, car: 100, 100, 200, 200, where "car" represents the target type label, and the following four numbers represent the target location information, namely the coordinates of the top-left corner vertex (x0, y0) and the bottom-right corner vertex (x1, y1) of the bounding box.

[0134] (3) Training the detection model

[0135] Figure 9 This is a flowchart illustrating the training process of the detection model. (For example...) Figure 9 As shown, the electronic device reads the sample images and corresponding target annotation information saved in steps (1) and (2) above. An initial detection model is constructed based on a small-scale detection model. The sample images are processed by the initial detection model to obtain the predicted location information of the target objects in the sample images. The intersection-union ratio of the location information is calculated using the target location information and the predicted location information in the target annotation information. That is, the intersection between the target location information and the predicted location information, and the union between the target location information and the predicted location information are calculated. Then, the ratio between the intersection and the union is calculated to obtain the intersection-union ratio.

[0136] The intersection-union ratio of the location information is set as the loss value. The model parameters of the initial detection model are updated iteratively using the backpropagation gradient of the loss value. Through continuous inference of the model and backpropagation of the loss value, the loss value is continuously reduced, which can continuously narrow the gap between the target location information and the predicted location information output by the initial detection model. The training process of the initial detection model is continuously run until the loss value is lower than the preset threshold, or after several rounds of iterative training, if the accuracy of the model does not improve, it means that the model has been trained to saturation. At this time, it can be considered that the model has been trained sufficiently and the trained model can be deployed to the device.

[0137] In this embodiment, rich sample images are first obtained based on the main prompt words of the target object, improving the diversity of the sample images. Then, the predicted annotation information of the sample images is detected by the initial detection model. The predicted annotation information and the target annotation information of the sample images are used to train the initial detection model, which not only improves the target detection effect of the trained target detection model, but also improves the efficiency of automatic annotation of sample images. The general large-scale image annotation model realizes automatic image generation and automatic labeling. It can continuously generate training data to optimize the small-scale target detection model, reducing the learning cost of the small-scale target detection model, and ensuring the detection accuracy of the small-scale target detection model.

[0138] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0139] Based on the same inventive concept, this application also provides a model training apparatus for implementing the model training method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more model training apparatus embodiments provided below can be found in the limitations of the model training method described above, and will not be repeated here.

[0140] In one embodiment, such as Figure 10 As shown, a model training device 1000 is provided, including: an image generation module 1001, a labeling and prediction module 1002, and a model training module 1003, wherein:

[0141] The image generation module 1001 is used to acquire a sample image corresponding to the target object; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object.

[0142] The annotation prediction module 1002 is used to input the sample image and the main prompt word into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image.

[0143] The model training module 1003 is used to iteratively train the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, so as to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

[0144] In one embodiment, the image generation module 1001 is further configured to obtain image generation values ​​corresponding to the main prompt word and the background prompt word of the target object; the background prompt word is used to describe the additional characteristics of the target object and / or the background information of its environment; the image generation value is used to control the generation of different images; and the main prompt word and the background prompt word are processed by image generation based on the image generation value through the text-based image model to obtain sample images corresponding to the main prompt word and the background prompt word.

[0145] In one embodiment, the model training device 1000 further includes a text-based image module, which is used to encode the main prompt word and the background prompt word through the text-based image model to obtain the main semantic information corresponding to the main prompt word and the background semantic information corresponding to the background prompt word; and to perform image generation processing on the main semantic information and the background semantic information based on the image generation value through the image generation model in the text-based image model to obtain the sample image corresponding to the main prompt word and the background prompt word.

[0146] In one embodiment, the model training device 1000 further includes an image annotation module for acquiring an image annotation model constructed based on the large-scale object detection model; the size of the image annotation model is larger than the size of the initial detection model; the large-scale object detection model is used to detect target objects in the image that match the input subject prompt words; through the image annotation model, based on the subject prompt words, target detection processing is performed on the sample image for the target objects to obtain the target type label and target location information of the target objects in the sample image; target annotation information is obtained according to the target type label and target location information.

[0147] In one embodiment, the model training device 900 further includes a position processing module, which is used to obtain combined annotation information based on the union of predicted position information in the predicted annotation information and target position information in the target annotation information; and to obtain identical annotation information based on the intersection of predicted position information and target position information.

[0148] In one embodiment, the model training module 1003 is further configured to determine the intersection-union ratio (IUR) between the predicted annotation information and the target annotation information by combining the annotation information and the same annotation information; obtain the loss value between the predicted annotation information and the target annotation information based on the IUR; and iteratively update the model parameters of the initial detection model based on the loss value until the preset training termination condition is met, thereby obtaining the target detection model.

[0149] In one embodiment, such as Figure 11 As shown, a target detection device 1100 is provided, including: an image acquisition module 1101 and a target detection module 1102, wherein:

[0150] Image acquisition module 1101 is used to acquire the image to be detected.

[0151] The target detection module 1102 is used to input the image to be detected into the target detection model, and to perform target detection processing on the image to be detected through the target detection model to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the above model training method.

[0152] The modules in the aforementioned model training device and target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0153] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a model training method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0154] Those skilled in the art will understand that Figure 12The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model training method, characterized in that, The method includes: A sample image corresponding to the target object is obtained; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object. The sample image and the main prompt word are input into the initial detection model to be trained. The initial detection model performs target detection processing on the sample image based on the main prompt word to obtain the predicted annotation information of the sample image. Based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, the initial detection model is iteratively trained to obtain a trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

2. The method according to claim 1, characterized in that, The acquisition of the sample image corresponding to the target object includes: Obtain image generation values ​​corresponding to the main prompt word and the background prompt word of the target object; the background prompt word is used to describe the additional characteristics of the target object and / or the background information of its environment; the image generation values ​​are used to control the generation of different images; Using the text-based image model, based on the image generation value, image generation processing is performed on the main prompt word and the background prompt word to obtain the sample image corresponding to the main prompt word and the background prompt word.

3. The method according to claim 2, characterized in that, The step of using a text-based image model to perform image generation processing on the main prompt and the background prompt based on the image generation value, to obtain the sample image corresponding to the main prompt and the background prompt, includes: The text-based image model is used to encode the main prompt words and the background prompt words to obtain the main semantic information corresponding to the main prompt words and the background semantic information corresponding to the background prompt words. Using the image generation model in the text-to-image model, based on the image generation value, image generation processing is performed on the subject semantic information and the background semantic information to obtain the sample image corresponding to the subject prompt word and the background prompt word.

4. The method according to claim 1, characterized in that, Before iteratively training the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information to obtain the trained target detection model, the method further includes: Obtain an image annotation model based on the large-scale object detection model; the size of the image annotation model is larger than the size of the initial detection model; the large-scale object detection model is used to detect target objects in the image that match the input subject prompt word; Using the image annotation model, based on the main prompt words, target detection processing is performed on the sample image for the target object to obtain the target type label and target location information of the target object in the sample image; The target annotation information is obtained based on the target type label and the target location information.

5. The method according to claim 4, characterized in that, Before iteratively training the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information to obtain the trained target detection model, the method further includes: The combined annotation information is obtained by combining the predicted location information in the predicted annotation information with the target location information in the target annotation information. The same annotation information is obtained based on the intersection between the predicted location information and the target location information.

6. The method according to claim 1, characterized in that, The step of iteratively training the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information to obtain a trained target detection model includes: For the combined annotation information and the identical annotation information, determine the intersection-union ratio between the predicted annotation information and the target annotation information; Based on the intersection-union ratio, the loss value between the predicted annotation information and the target annotation information is obtained; Based on the loss value, the model parameters of the initial detection model are iteratively updated until the preset training termination condition is met, thus obtaining the target detection model.

7. A target detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into the target detection model, and the target detection model performs target detection processing on the image to be detected to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in any one of claims 1 to 6.

8. A model training device, characterized in that, The device includes: An image generation module is used to acquire a sample image corresponding to a target object; the sample image is obtained based on the main prompt words of the target object; the main prompt words are used to describe the characteristics of the target object. The annotation prediction module is used to input the sample image and the main prompt word into the initial detection model to be trained, and to perform target detection processing on the sample image based on the main prompt word through the initial detection model to obtain the predicted annotation information of the sample image; The model training module is used to iteratively train the initial detection model based on the combined annotation information and identical annotation information between the predicted annotation information and the target annotation information of the sample image, so as to obtain the trained target detection model; the target annotation information is obtained by annotating the target objects in the sample image.

9. A target detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected; The target detection module is used to input the image to be detected into the target detection model, and to perform target detection processing on the image to be detected through the target detection model to obtain the target detection result of the image to be detected for the target object; the target detection model is trained by the method described in any one of claims 1 to 6.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6 or 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6 or 7.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6 or 7.

Images