Object detection method and device, electronic equipment, storage medium and product

Abstract

The invention provides an object detection method and device, electronic equipment, a storage medium and a product, and relates to the technical field of computer vision. The method comprises the steps of obtaining a to-be-detected image and a description text; determining a to-be-detected image area occupied by a to-be-detected object in the to-be-detected image by using the position prediction model; inputting each to-be-detected image area and the description text into an image-text matching model to obtain a matching degree between the description text and each to-be-detected image area; selecting an image area with the maximum matching degree with the description text from the alternative image areas as an image area matched with the description text; the alternative image area is determined from the to-be-detected image areas. According to the method, the position prediction model and the image-text matching model can be trained with relatively low labor cost and hardware performance requirements, so that the use and development cost of open word detection is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an object detection method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] Open-Vocabulary Object Detection (OVOD) is a new technology in the field of artificial intelligence. Its goal is to use manually inputted prompts to guide the OVOD model in locating the objects indicated by those prompts in images acquired from the open world. Compared with traditional detection tasks and algorithms, OVOD models have a wider range of downstream applications and are more flexible in deployment. However, OVOD models also place higher demands on the computational resources required for training, as well as the size and quality of the training dataset.

[0003] In existing technologies, training OVOD models typically relies on a large training dataset, which needs to contain a sufficiently rich image library, a descriptive text library, and the location and descriptive text of each object in the image.

[0004] However, to ensure the OVOD model can accurately locate objects in images, the available open-source training datasets, which simultaneously annotate object locations and descriptive text, are relatively small and insufficient to support OVOD model training. Therefore, current technologies require significant manpower to annotate the location and description of each object in a wide variety of images. Furthermore, when adjusting the OVOD model parameters based on the loss value, the accuracy of both object location and category must be considered simultaneously, resulting in numerous training iterations and consuming substantial hardware resources. Thus, current technologies for training OVOD models are labor-intensive and demanding in terms of hardware performance, leading to high costs for the use and development of open word detection. Summary of the Invention

[0005] The purpose of this invention is to provide an object detection method, apparatus, electronic device, storage medium, and product to train a location prediction model and an image-text matching model with lower labor costs and hardware performance requirements, thereby achieving the functionality of the existing OVOD model and reducing the use and development costs of open word detection. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide an object detection method, the method comprising:

[0007] Obtain the image to be detected and the descriptive text to be matched;

[0008] Using a pre-trained location prediction model, the image region occupied by each object to be detected in the image to be detected is determined as the image region to be detected; wherein, the location prediction model is trained using a first sample image and the real positions of sample objects in the first sample image;

[0009] Each image region to be detected and the descriptive text are input into a pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected; wherein, the image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image;

[0010] The image region with the highest degree of matching with the descriptive text is selected from the candidate image regions and used as the image region in the image to be detected that matches the descriptive text; wherein, the candidate image regions are determined from each image region to be detected.

[0011] Optionally, the location prediction model is trained through the following steps:

[0012] Obtain the first sample image and the true location of each sample object in the first sample image;

[0013] The first sample image is input into the location prediction model of the initial structure to obtain the predicted location of the object in the first sample image;

[0014] The loss value is calculated based on the difference between the predicted location and the actual location;

[0015] The network parameters of the initial location prediction model are adjusted based on the loss value until the first convergence condition is met, thus obtaining the location prediction model.

[0016] Optionally, there may be multiple predicted locations;

[0017] The calculation of the loss value based on the difference between the predicted location and the actual location includes:

[0018] For each first sample image, if there are multiple sample objects in the first sample image, then for each sample object in the first sample image, calculate the intersection-union ratio between the true position of the sample object and each predicted position in the first sample image;

[0019] The predicted location with the largest intersection-union ratio (IU) with the actual location of the sample object is determined as the location to be utilized for that sample object.

[0020] The loss value is calculated based on the difference between the actual location of each sample object and the corresponding location to be utilized.

[0021] Optionally, the image-text matching model is trained through the following steps:

[0022] Obtain the second sample image and the actual descriptive text of the second sample image;

[0023] The second sample image and its actual descriptive text are input into the image-text matching model of the initial structure to obtain the degree of matching between the second sample image and its actual descriptive text.

[0024] Based on the degree of matching between the second sample image and the real descriptive text of the second sample image, the network parameters of the initial image-text matching model are adjusted until the second convergence condition is met, thus obtaining the image-text matching model.

[0025] Optionally, the step of inputting each image region to be detected and the descriptive text into a pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected includes:

[0026] The negative prompt text, each image region to be detected, and the description text are input into a pre-trained image-text matching model to obtain the matching degree between the description text and each image region to be detected, as well as the matching degree between the negative prompt text and each image region to be detected.

[0027] Before selecting the image region from the candidate image regions that best matches the descriptive text, the method further includes:

[0028] From each image region to be detected, image regions whose matching degree with the negative prompt text is no greater than a preset threshold are determined as candidate image regions.

[0029] Secondly, embodiments of the present invention provide an object detection device, the device comprising:

[0030] The first acquisition module is used to acquire the image to be detected and the descriptive text to be matched;

[0031] The module is used to determine the image region occupied by each object to be detected in the image to be detected using a pre-trained location prediction model, which is then used as the image region to be detected; wherein, the location prediction model is trained using a first sample image and the real positions of sample objects in the first sample image.

[0032] The input module is used to input each image region to be detected and the descriptive text into a pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected; wherein, the image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image;

[0033] The selection module is used to select the image region with the highest degree of matching with the descriptive text from the candidate image regions, and use it as the image region in the image to be detected that matches the descriptive text; wherein, the candidate image regions are determined from each image region to be detected.

[0034] Optionally, the location prediction model is trained through the following steps:

[0035] Obtain the first sample image and the true location of each sample object in the first sample image;

[0036] The first sample image is input into the location prediction model of the initial structure to obtain the predicted location of the object in the first sample image;

[0037] The loss value is calculated based on the difference between the predicted location and the actual location;

[0038] Based on the loss value, the network parameters of the initial structure location prediction model are adjusted until the first convergence condition is met, thus obtaining the location prediction model.

[0039] or,

[0040] The predicted locations are multiple;

[0041] The calculation of the loss value based on the difference between the predicted location and the actual location includes:

[0042] For each first sample image, if there are multiple sample objects in the first sample image, then for each sample object in the first sample image, calculate the intersection-union ratio between the true position of the sample object and each predicted position in the first sample image;

[0043] The predicted location with the largest intersection-union ratio (IU) with the actual location of the sample object is determined as the location to be utilized for that sample object.

[0044] The loss value is calculated based on the difference between the actual location of each sample object and the corresponding location to be utilized.

[0045] And / or,

[0046] The image-text matching model is trained through the following steps:

[0047] Obtain the second sample image and the actual descriptive text of the second sample image;

[0048] The second sample image and its actual descriptive text are input into the image-text matching model of the initial structure to obtain the degree of matching between the second sample image and its actual descriptive text.

[0049] Based on the degree of matching between the second sample image and the real description text of the second sample image, the network parameters of the initial image-text matching model are adjusted until the second convergence condition is met, and the image-text matching model is obtained.

[0050] And / or,

[0051] The input module is specifically used to input the negative prompt text, each image region to be detected, and the description text into a pre-trained image-text matching model to obtain the matching degree between the description text and each image region to be detected, as well as the matching degree between the negative prompt text and each image region to be detected.

[0052] The device further includes:

[0053] The determination module is used to determine, from each image region to be detected, image regions whose matching degree with the negative prompt text is no greater than a preset threshold, as candidate image regions.

[0054] Thirdly, embodiments of the present invention also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0055] Memory, used to store computer programs;

[0056] The processor, when executing a program stored in memory, implements the object detection method provided in the first aspect.

[0057] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the object detection method provided in the first aspect.

[0058] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the object detection method provided in the first aspect.

[0059] Beneficial effects of the embodiments of the present invention:

[0060] In this embodiment of the invention, since the pre-trained location prediction model is trained using the first sample image and the real location of the sample objects in the first sample image, the location prediction model can identify the location of each object to be detected in the input image to be detected, and thus determine the image area occupied by each object to be detected; since the pre-trained image-text matching model is trained using the second sample image and the real descriptive text representing the second sample image, the image-text matching model can identify the degree of matching between each input image area to be detected and the descriptive text.

[0061] In the object detection method provided by this invention, the location prediction model only needs to be trained on the first sample image and the real locations of each object to be detected within it, and the image-text matching model only needs to be trained on the second sample image and its real descriptive text. The training data required for both the location prediction and image-text matching models are available in open-source datasets. When training these two models, the open-source datasets can be directly reused, eliminating the need for manpower to label the location and descriptive text of each object in a variety of images. Furthermore, the location prediction model focuses on optimizing the location prediction loss during training, while the image-text matching model focuses on optimizing the image-text matching loss. Each model has a single and clear training objective, resulting in faster convergence speed, reduced training iterations, and lower hardware costs during training. This invention, by separating object localization and object classification into two independent models, trains the location prediction model and image-text matching model with lower manpower costs and hardware performance requirements to achieve open word detection, reducing the usage and development costs of open word detection.

[0062] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0064] Figure 1 A flowchart illustrating the first object detection method provided in an embodiment of the present invention;

[0065] Figure 2 This is a flowchart illustrating the second object detection method provided in an embodiment of the present invention;

[0066] Figure 3 This is a flowchart illustrating the third object detection method provided in an embodiment of the present invention.

[0067] Figure 4 This is a schematic diagram illustrating the principle of a location prediction model in an object detection method provided in an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram illustrating the effect of a location prediction model in an object detection method provided by an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram illustrating the principle of the image-text matching model in an object detection method provided in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the structure of an object detection device provided in an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0073] In order to train a location prediction model and an image-text matching model with lower labor costs and hardware performance requirements, so as to realize the function of the existing OVOD model and reduce the use and development costs of open word detection, this invention provides an object detection method, device, electronic device, storage medium and product.

[0074] The object detection method provided in this embodiment of the invention can be applied to electronic devices, such as servers, desktop computers, or laptops.

[0075] An object detection method provided in this embodiment of the invention includes:

[0076] Obtain the image to be detected and the descriptive text to be matched;

[0077] Using a pre-trained location prediction model, the image region occupied by each object to be detected in the image to be detected is determined as the image region to be detected; wherein, the location prediction model is trained using a first sample image and the real positions of sample objects in the first sample image;

[0078] Each image region to be detected and its descriptive text are input into a pre-trained image-text matching model to obtain the degree of matching between the descriptive text and each image region to be detected. The image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image.

[0079] The image region with the highest degree of matching with the descriptive text is selected from the candidate image regions and used as the image region in the image to be detected that matches the descriptive text; wherein, the candidate image regions are determined from each image region to be detected.

[0080] In the object detection method provided by this invention, the location prediction model only needs to be trained on the first sample image and the real locations of each object to be detected within it, and the image-text matching model only needs to be trained on the second sample image and its real descriptive text. The training data required for both the location prediction and image-text matching models are available in open-source datasets. When training these two models, the open-source datasets can be directly reused, eliminating the need for manpower to label the location and descriptive text of each object in a variety of images. Furthermore, the location prediction model focuses on optimizing the location prediction loss during training, while the image-text matching model focuses on optimizing the image-text matching loss. Each model has a single and clear training objective, resulting in faster convergence speed, reduced training iterations, and lower hardware costs during training. This invention, by separating object localization and object classification into two independent models, trains the location prediction model and image-text matching model with lower manpower costs and hardware performance requirements to achieve open word detection, reducing the usage and development costs of open word detection.

[0081] The following describes an object detection method provided by an embodiment of the present invention with reference to the accompanying drawings. Figure 1 As shown, the method may include steps S101-S104.

[0082] S101, Obtain the image to be detected and the descriptive text to be matched.

[0083] It is understood that the image to be detected can be an image captured by an image acquisition device, an image downloaded from a media website, or a virtual image generated through artificial intelligence (AI) technology. For example, the image to be detected may contain objects of known and / or unknown categories, wherein the known categories may be categories used when training the image-text matching model in this embodiment of the invention, and the unknown categories may be categories not used when training the image-text matching model in this embodiment of the invention. At least one image to be detected may exist, and the electronic device may, for each image to be detected, determine an image region in that image that matches the descriptive text.

[0084] For example, an electronic device can receive a detection command sent by a user, which carries an image to be detected and descriptive text to be matched. The electronic device can obtain the image to be detected and the descriptive text from the detection command. In another implementation, the detection command does not carry an image to be detected or descriptive text to be matched, but carries the storage location of the image to be detected and the storage location of the descriptive text; the electronic device can obtain the image to be detected and the descriptive text according to the storage location indicated by the detection command. The methods for obtaining the image to be detected and the descriptive text are only illustrative examples and are not specifically limited in this embodiment of the invention.

[0085] It is understandable that when a user needs to determine the location of the object described in the descriptive text in an image to be detected, the object detection method provided in this embodiment of the invention can be applied. Both the descriptive text and the image to be detected can be selected by the user, and the user is unaware whether the object described in the descriptive text was used when training the image-text matching model, or whether the image to be detected contains the object described in the descriptive text.

[0086] The descriptive text describes the category to be matched against content in the image. The category described in the descriptive text can be a known category or an unknown category. For example, the descriptive text can be natural language text, such as at least one of "person in red clothes," "white cat," and "person walking a dog."

[0087] There can be only one descriptive text, and the electronic device can determine all image regions in the image to be detected that match the descriptive text; there can be multiple descriptive texts, and the electronic device can determine all image regions in the image to be detected that match each descriptive text.

[0088] S102, using a pre-trained location prediction model, determines the image region occupied by each object to be detected in the image to be detected, which is then used as the image region to be detected.

[0089] The location prediction model is trained using the first sample image and the actual locations of the sample objects in the first sample image.

[0090] Understandably, an electronic device can input an image to be detected into a location prediction model. The model can then determine a bounding box containing the rectangle of each object to be detected, thus obtaining the image region to be detected. Each bounding box can contain the object being detected. The model then outputs the diagonal coordinates of the bounding boxes for each object; for example, the model can output the top-left and bottom-right coordinates of the bounding box for a single object. There can be multiple image regions to be detected. For instance, if the image contains multiple objects, the location prediction model can determine the image region to be detected for each object.

[0091] Understandably, a location prediction model can obtain the coordinates of all objects in an image to be detected, thereby determining the image region of each object. Assuming an image constitutes a geometric plane, with the top-left corner of the image as the origin (0,0), and the y-axis pointing downwards and the x-axis pointing to the right, a Cartesian coordinate system is established. The image region of each object in the image can be represented as (x1, y1, x2, y2), where (x1, y1) is the coordinate of the top-left corner of the object's image region, and (x2, y2) is the coordinate of the bottom-right corner of the object's image region. The output of the location prediction model can be expressed as follows:

[0092]

[0093] Where, θ i Let represent the input image containing i objects, and σ represent the set of coordinates of all objects in the input image. This represents the coordinates of the top-left corner of the i-th object in the input image. f(θ) represents the coordinates of the bottom right corner of the i-th object in the input image. i ) represents the location prediction model.

[0094] Each image region to be detected can correspond to an object, and the objects corresponding to each image region are different. Subsequently, for each image region to be detected, the degree of matching between that image region and the descriptive text can be determined; that is, the probability that the object corresponding to that image region belongs to the category described in the descriptive text can be determined. In this embodiment of the invention, by using one image region to be detected as one object, the image-text matching model can easily determine the degree of matching between an object and a category directly, without requiring the image-text matching model to obtain the position of each object in the image to be detected.

[0095] The location prediction model is trained using a first sample image and the actual locations of the sample objects within that image. For example, during training, the first sample image can be input into the initial structure of the location prediction model to obtain the predicted locations of each object. Based on the difference between the predicted locations and the actual locations of the sample objects in the first sample image, a loss value is calculated, and then the network parameters of the location prediction model are adjusted based on the loss value. The location prediction model can be a bounding box acquisition network, such as a Region-based Convolutional Neural Network (RCNN) or a Universal Proposal Network (UPN). In this embodiment of the invention, an open-source, pre-trained location prediction model can be directly used.

[0096] For example, the location prediction model is the UPN network, which is a Detection Transformer (DETR) network that introduces a deformable attention mechanism. Based on this mechanism, the UPN network can dynamically adjust the image region of interest according to the input content, more flexibly capturing the features of key targets in the image. The workflow of the UPN network is as follows: First, the image to be detected is converted into image features, then processed by the decoder in the UPN network, outputting a series of vectors of shape (n, l), where n represents the total number of output results and l represents the length of the output. The specific components of the output vector include: four-dimensional coordinates (corresponding to the diagonal coordinates of the selection box), one-dimensional object classification information, and one-dimensional confidence. Since the object classification result is not needed, l is actually 5-dimensional. The confidence score is used to filter the results: if the confidence score reaches a threshold, the object location result is accepted. In practical applications, n is usually set to 900, meaning that a single image generates a maximum of 900 object location results. Then, these 900 location results are filtered by a confidence threshold, retaining the location results with higher confidence.

[0097] In one implementation, the location prediction model is trained through the following steps:

[0098] A1, obtain the first sample image and the true location of each sample object in the first sample image.

[0099] A2, input the first sample image into the position prediction model of the initial structure to obtain the predicted position of the object in the first sample image.

[0100] A3. Calculate the loss value based on the difference between the predicted location and the actual location.

[0101] A4. Based on the loss value, the network parameters of the initial structure location prediction model are adjusted until the first convergence condition is met, thus obtaining the location prediction model.

[0102] It is understandable that the electronic device used to train the location prediction model and the electronic device used to perform the object detection method can be different devices, and the electronic device used to perform the object detection method can also be used to train the location prediction model. The electronic device can obtain the first sample image and the true location of each sample object in the first sample image from an open-source dataset. The open-source dataset contains a large number of images and the locations of the bounding boxes of each object in the images. The open-source dataset can be the Common Objects in Context (COCO) dataset or the Pattern Analysis, Statistical Modelling and Computational Learning (PASCAL) dataset.

[0103] The electronic device can input the first sample image into the location prediction model of the initial structure to obtain the predicted position of each object in the first sample image output by the location prediction model of the initial structure. Then, based on the difference between the obtained predicted position and the true position, the loss value is calculated. Based on the loss value, the network parameters of the location prediction model of the initial structure are adjusted until the first convergence condition is reached to obtain the location prediction model.

[0104] Because adjusting the network parameters of the location prediction model makes the predicted location output by the adjusted network parameters closer to the actual location, the trained location prediction model can output accurate predicted locations, and the determined image regions of each object to be detected are accurate.

[0105] The first convergence condition is the criterion for determining whether model training has ended. For example, the first convergence condition could be that the loss value is less than a threshold, the loss value decreases by less than a threshold after a specified number of consecutive training epochs, or the maximum number of training epochs has been reached. This embodiment of the invention is merely illustrative and does not constitute a specific limitation.

[0106] In one implementation, there are multiple predicted positions; step A3 includes steps A31-A33.

[0107] A31, for each first sample image, if there are multiple sample objects in the first sample image, then for each sample object in the first sample image, calculate the intersection-union ratio between the true position of the sample object and each predicted position of the first sample image.

[0108] It is understandable that if there are multiple sample objects in a first sample image, there are multiple true locations for each sample object and multiple predicted locations. Therefore, when calculating the loss value, it is necessary to determine the true location corresponding to each predicted location.

[0109] For each sample object in a first sample image, the electronic device can calculate the intersection-union ratio (IUR) between the true position and each predicted position of the sample object. For example, the overlapping area of ​​the true position and each predicted position of the sample object in the image to be detected is calculated, and the calculated overlapping area is divided by the total area occupied by the sample object and the predicted position in the image to be detected to obtain the IUR between the sample object and the predicted position.

[0110] A32, determine the predicted position with the largest intersection-union ratio (IU) with the actual position of the sample object, and use it as the position to be utilized for the sample object.

[0111] Understandably, for each sample object in a first sample image, the electronic device can determine the predicted position with the highest intersection-over-union ratio (IoU) with the true position of that sample object (i.e., the position that overlaps the most with the true position), and use the determined predicted position as the location to be utilized corresponding to the true position of that sample object. For example, if the IoU of predicted position 1 with the true position is 80%, and the IoU of predicted position 2 with the true position is 30%, then it can be determined that the object contained in predicted position 1 and the true position may be the same object, and predicted position 1 is determined as the location to be utilized.

[0112] A33. Calculate the loss value based on the difference between the actual location of each sample object and the corresponding location to be utilized.

[0113] Understandably, electronic devices can calculate the difference between the true location of each sample object and its corresponding location to be utilized. For example, the difference between the top-left coordinate of the true location of each sample object and the top-left coordinate of its corresponding location to be utilized, as well as the difference between the bottom-right coordinate of the true location of the sample object and the bottom-right coordinate of its corresponding location to be utilized, are calculated. Combining these two differences (e.g., calculating the sum of squares or the average of these differences), the difference between the predicted location and the true location of the sample object is obtained. Then, the differences of all sample objects are summed (e.g., calculating the sum of squares or the average of all differences) to obtain a loss value that represents the difference between the predicted and true results for all sample objects in the entire first sample image. The loss value is inversely correlated with the accuracy of the location prediction model.

[0114] S103, input each image region to be detected and the descriptive text into the pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected.

[0115] The image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image.

[0116] Understandably, electronic devices can input each image region to be detected and its descriptive text into a pre-trained image-text matching model. This model includes an image encoder and a text encoder. The image encoder encodes each input image region into an image feature vector, and the text encoder encodes the input descriptive text into a text feature vector. Then, the similarity between the image feature vector and the text feature vector is calculated to obtain the degree of matching between the descriptive text and each image region to be detected. Specifically, the electronic device uses the image-text matching model to obtain the image feature vector of each image region to be detected and the text feature vector of each descriptive text, thereby determining the degree of matching between the detected image region and the descriptive text.

[0117] For example, the cosine similarity between the image feature vector and the text feature vector can be calculated to obtain the degree of matching between the descriptive text and each image region to be detected; where the cosine similarity is positively correlated with the degree of matching. In another implementation, the cross-entropy loss between the image feature vector and the text feature vector can be calculated to obtain the degree of matching between the descriptive text and each image region to be detected; where the cross-entropy loss is negatively correlated with the degree of matching.

[0118] The image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image. For example, during training, the second sample image and the real descriptive text can be input into the initial structure of the image-text matching model to obtain the predicted matching degree between the second sample image and the real descriptive text. Since the real descriptive text represents the true category of the second sample image, the true matching degree between the second sample image and the real descriptive text indicates a perfect match, for example, a true matching degree of 1. When adjusting the network parameters of the image-text matching model, the predicted matching degree can be made close to the true matching degree; that is, the predicted matching degree is used as the loss value to adjust the network parameters of the image-text matching model. The image-text matching model can be a Contrastive Language Image Pre-training (CLIP) model. In this embodiment of the invention, an open-source trained image-text matching model can be used directly.

[0119] Understandably, for information in two modalities—descriptive text and image regions—electronic devices can input them separately into two independent encoders in the image-text matching model: a text encoder for processing descriptive text and an image encoder for processing image regions. This process can be mathematically represented as: αi =f T (t), b i =f I (x). Where f T f represents a text encoder. I This represents the image encoder, t represents the descriptive text, x represents the image region, and α represents the image encoder. i Let b represent the feature vector of the i-th descriptive text. i This represents the feature vector of the i-th image region.

[0120] In actual training, the core objective of the image-text matching model is to align these two feature vectors, that is, to make text and image features belonging to the same category more similar. Therefore, the core objective of the image-text matching model can be expressed as: minCE(α) i ·b i , l i Where CE represents the cross-entropy loss, minCE represents the minimum cross-entropy loss, and l i The true value label is usually 1. The cross-entropy loss can be used to represent the degree of matching between the image region and the descriptive text.

[0121] In practical applications, electronic devices can input descriptive text into the text encoder of the image-text matching model for encoding, resulting in a text feature vector of size (n, A), where n represents the total number of descriptive texts and A represents the dimension of the text feature vector. In open word classification scenarios, the dimensions of the image feature vector and the text feature vector need to be consistent (both are A-dimensional). The electronic device can input the image region to be detected into the image encoder of the image-text matching model, encoding it to obtain an image feature vector b of dimension A. Then, by performing matrix multiplication between the text feature matrix (n, A) and the image feature vector b (A, 1), the matching degree between the image region and all descriptive texts can be calculated at once. This process can be represented dimensionally as: (n, A) × A → n. The resulting n-dimensional result vector (each element corresponding to the matching degree between a descriptive text and the image region) is activated by an activation function (softmax), transforming it into a probability distribution corresponding to each descriptive text. Finally, the result of open word classification (i.e., the descriptive text most likely to match the image region) can be obtained through this probability distribution.

[0122] In one implementation, the image-text matching model is trained through the following steps:

[0123] B1, obtain the second sample image and the actual description text of the second sample image.

[0124] B2, input the second sample image and the real description text of the second sample image into the image-text matching model of the initial structure, and obtain the matching degree between the second sample image and the real description text of the second sample image.

[0125] B3. Based on the degree of matching between the second sample image and the real description text of the second sample image, the network parameters of the initial image-text matching model are adjusted until the second convergence condition is met, thus obtaining the image-text matching model.

[0126] It is understandable that the electronic device used to train the image-text matching model and the electronic device used to perform the object detection method can be different devices, and the electronic device used to perform the object detection method can also be used to train the image-text matching model. The electronic device can obtain second sample images and their ground truth descriptive text from an open-source dataset. This open-source dataset contains a large number of images and ground truth descriptive text representing the image categories. The open-source dataset can be the COCO dataset or the ImageNet dataset.

[0127] The electronic device can input a second sample image and real descriptive text into an initial-structure image-text matching model to obtain the matching degree between the second sample image and the real descriptive text output by the initial-structure image-text matching model. Then, based on the matching degree, the network parameters of the initial-structure location prediction model are adjusted until a second convergence condition is met, thus obtaining the image-text matching model. For example, the second convergence condition could be a matching degree greater than a threshold, a matching degree increasing by less than a threshold after a specified number of consecutive training epochs, or reaching the maximum number of training epochs. This embodiment of the invention is merely illustrative and does not constitute a specific limitation.

[0128] It is understood that electronic devices can acquire new second sample images and their real descriptive text to train image-text matching models when training is required. For example, at the current moment, upon receiving a user's training instruction or reaching a preset periodic time, new second sample images and their real descriptive text are acquired to train the image-text matching model. These new second sample images and their real descriptive text may not exist in the second sample images and their real descriptive text used in previous image-text matching model training. Specifically, the electronic device can acquire new real descriptive text and then use web crawling technology to acquire the second sample image described by the new real descriptive text. Since the image-text matching model used in this embodiment only requires the image and its descriptive text for training, and does not require the location of the object described by the descriptive text in the image, training samples can be obtained solely through web crawling technology, eliminating the need for manual annotation of the location of the object described by the descriptive text, thus reducing the manual cost of sample acquisition.

[0129] S104: Select the image region with the highest degree of matching with the descriptive text from the candidate image regions, and use it as the image region in the image to be detected that matches the descriptive text.

[0130] Among them, the candidate image regions are determined from each image region to be detected.

[0131] Understandably, electronic devices can determine candidate image regions from each image region to be detected, and then select the image region with the highest degree of matching with the descriptive text from the candidate image regions as the image region in the image to be detected that matches the descriptive text.

[0132] In one implementation, the electronic device can use each image region to be detected as a candidate image region, that is, select the image region with the highest degree of matching with the descriptive text from all the image regions to be detected. In another implementation, the electronic device can determine the candidate image regions from each image region to be detected based on the negative prompt text; this implementation will be described in detail in subsequent embodiments.

[0133] In the object detection method provided by this invention, the location prediction model only needs to be trained on the first sample image and the real locations of each object to be detected within it, and the image-text matching model only needs to be trained on the second sample image and its real descriptive text. The training data required for both the location prediction and image-text matching models are available in open-source datasets. When training these two models, the open-source datasets can be directly reused, eliminating the need for manpower to label the location and descriptive text of each object in a variety of images. Furthermore, the location prediction model focuses on optimizing the location prediction loss during training, while the image-text matching model focuses on optimizing the image-text matching loss. Each model has a single and clear training objective, resulting in faster convergence speed, reduced training iterations, and lower hardware costs during training. This invention, by separating object localization and object classification into two independent models, trains the location prediction model and image-text matching model with lower manpower costs and hardware performance requirements to achieve open word detection, reducing the usage and development costs of open word detection.

[0134] In this embodiment of the invention, the location prediction model and the image-text matching model are trained separately. If the location prediction model or the image-text matching model fails to meet the performance standards (e.g., the accuracy is not up to standard) based on the test set, the location prediction model or the image-text matching model can be retrained. In another implementation, the base model used by the location prediction model or the image-text matching model can be replaced with another model.

[0135] In one embodiment, in Figure 1 Based on the object detection method shown, such as Figure 2 As shown, step S103 includes step S1031, and the method further includes step S201.

[0136] S1031, input the negative prompt text, each image region to be detected, and the descriptive text into the pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected, as well as the matching degree between the negative prompt text and each image region to be detected.

[0137] S201, from each image region to be detected, determine the image region whose matching degree with the negative prompt text is no greater than a preset threshold, and use it as the candidate image region.

[0138] Understandably, location prediction models can identify the regions of all objects in an image to be detected. However, some objects do not require category detection. These objects may be meaningless, such as those not belonging to the categories labeled in the COCO dataset. While these objects may resemble the categories described in the descriptive text, they do not actually belong to those categories. The degree of match between these objects and the descriptive text may be high, leading to the incorrect identification of the image region containing these objects as belonging to the category described in the text. In other words, the electronic device may mistakenly associate these objects with the descriptive text, affecting the accuracy of object detection. For example, if the object is a basketball, but the descriptive text describes a volleyball, the electronic device will incorrectly associate the basketball with the descriptive text.

[0139] To improve the accuracy of the image-text matching model, the negative prompt text can be a word unrelated to the descriptive text, such as "object" or "thing". The electronic device can input the negative prompt text, each image region to be detected, and the descriptive text into the image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected, as well as the matching degree between the negative prompt text and each image region to be detected.

[0140] Electronic devices can identify image regions from each image region to be detected whose matching degree with the negative prompt text is no greater than a preset threshold, and use these as candidate image regions. The preset threshold can be 0.3 or 0.2, etc. If the matching degree of an image region to be detected with the negative prompt text is greater than the preset threshold, the object in that image region can be determined as a meaningless object. Even if the image region to be detected has a high similarity to the descriptive text, the object in that image region may only have some similarity to the object described in the descriptive text; the object itself is still meaningless. Electronic devices can use this method to filter out image regions to be detected containing meaningless objects, and determine image regions whose matching degree with the negative prompt text is no greater than the preset threshold as candidate image regions, thereby improving the accuracy of the identified image regions that match the descriptive text.

[0141] Figure 3This is a flowchart illustrating the third object detection method provided in this embodiment of the invention, as shown below. Figure 3 As shown, the object detection method includes steps S301-S304.

[0142] S301, Regression network to obtain target bounding boxes.

[0143] Understandably, the location prediction model is a regression network. Electronic devices can use a pre-trained location prediction model to determine the image region occupied by each object to be detected in the image to be detected, and obtain the target box as the image region to be detected.

[0144] S302, Get the small image of the target bounding box.

[0145] Understandably, electronic devices can crop out target bounding boxes from the image to be detected, obtaining individual smaller images, that is, the image area occupied by each object to be detected.

[0146] S303, the small image uses an open classifier.

[0147] It is understandable that the image-text matching model is an open classifier. Electronic devices can input each image region to be detected and the descriptive text into the image-text matching model to obtain the degree of matching between the descriptive text and each image region to be detected.

[0148] S304 combines the bounding box with the classifier to obtain the result.

[0149] Understandably, electronic devices can select the image region with the highest degree of matching with the descriptive text from the candidate image regions as the image region in the image to be detected that matches the descriptive text, thereby combining the bounding box with the classifier to obtain the result; whereby the candidate image regions are determined from each image region to be detected.

[0150] In the object detection method provided by this invention, the location prediction model only needs to be trained on the first sample image and the real locations of each object to be detected within it, and the image-text matching model only needs to be trained on the second sample image and its real descriptive text. The training data required for both the location prediction and image-text matching models are available in open-source datasets. When training these two models, the open-source datasets can be directly reused, eliminating the need for manpower to label the location and descriptive text of each object in a variety of images. Furthermore, the location prediction model focuses on optimizing the location prediction loss during training, while the image-text matching model focuses on optimizing the image-text matching loss. Each model has a single and clear training objective, resulting in faster convergence speed, reduced training iterations, and lower hardware costs during training. This invention, by separating object localization and object classification into two independent models, trains the location prediction model and image-text matching model with lower manpower costs and hardware performance requirements to achieve open word detection, reducing the usage and development costs of open word detection.

[0151] Figure 4 This is a schematic diagram illustrating the principle of a location prediction model in an object detection method provided by an embodiment of the present invention, as shown below. Figure 4 As shown, the electronic device can input an image containing two birds into the location prediction model's convolutional neural network (CNN) to extract features, obtaining a set of image features. Then, the image feature set is input into the transformer encoder-decoder in the location prediction model to generate a set of box predictions. Next, bipartite matching loss is calculated. This process matches the set of box predictions with cases where there are no objects, filtering out cases where there are no objects in the set of box predictions, thus more accurately determining the image regions of each object in the image to be detected.

[0152] Figure 5 This is a schematic diagram illustrating the effect of a location prediction model in an object detection method provided by an embodiment of the present invention, as shown below. Figure 5 As shown, the image to be detected is a picture taken in a parking lot. The objects in the image include trees, cars, and road signs (as shown by the dashed box in the figure). The location prediction model can output the coordinates of the dashed box of each object in the image to be detected.

[0153] Figure 6This is a schematic diagram illustrating the principle of the image-text matching model in an object detection method provided by an embodiment of the present invention. The electronic device can use the text encoder in the image-text matching model to encode the descriptive text, obtaining a text feature sequence T1, T2, T3, ..., T... representing the descriptive text. N Electronic devices can use the image encoder in the image-text matching model to encode image regions, obtaining image feature sequences I1, I2, I3, ..., I... representing the image regions. N Then, the similarity between the text feature sequence and the image feature sequence is calculated.

[0154] Figure 7 This is a schematic diagram of the structure of an object detection device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the device includes:

[0155] The first acquisition module 710 is used to acquire the image to be detected and the descriptive text to be matched;

[0156] Module 720 is used to determine the image region occupied by each object to be detected in the image to be detected using a pre-trained position prediction model, which is then used as the image region to be detected; wherein, the position prediction model is trained using a first sample image and the real positions of sample objects in the first sample image.

[0157] The input module 730 is used to input each image region to be detected and the descriptive text into a pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected; wherein, the image-text matching model is trained using a second sample image and the real descriptive text representing the second sample image;

[0158] The selection module 740 is used to select the image region with the highest degree of matching with the descriptive text from the candidate image regions, and use it as the image region in the image to be detected that matches the descriptive text; wherein, the candidate image regions are determined from each image region to be detected.

[0159] Optionally, the location prediction model is trained through the following steps:

[0160] Obtain the first sample image and the true location of each sample object in the first sample image;

[0161] The first sample image is input into the location prediction model of the initial structure to obtain the predicted location of the object in the first sample image;

[0162] The loss value is calculated based on the difference between the predicted location and the actual location;

[0163] Based on the loss value, the network parameters of the initial structure location prediction model are adjusted until the first convergence condition is met, thus obtaining the location prediction model.

[0164] Optionally, there may be multiple predicted locations;

[0165] The calculation of the loss value based on the difference between the predicted location and the actual location includes:

[0166] For each first sample image, if there are multiple sample objects in the first sample image, then for each sample object in the first sample image, calculate the intersection-union ratio between the true position of the sample object and each predicted position in the first sample image;

[0167] The predicted location with the largest intersection-union ratio (IU) with the actual location of the sample object is determined as the location to be utilized for that sample object.

[0168] The loss value is calculated based on the difference between the actual location of each sample object and the corresponding location to be utilized.

[0169] Optionally, the image-text matching model is trained through the following steps:

[0170] Obtain the second sample image and the actual descriptive text of the second sample image;

[0171] The second sample image and its actual descriptive text are input into the image-text matching model of the initial structure to obtain the degree of matching between the second sample image and its actual descriptive text.

[0172] Based on the degree of matching between the second sample image and the real description text of the second sample image, the network parameters of the initial image-text matching model are adjusted until the second convergence condition is met, and the image-text matching model is obtained.

[0173] Optionally, the input module 730 is specifically used to input the negative prompt text, each image region to be detected, and the descriptive text into a pre-trained image-text matching model to obtain the matching degree between the descriptive text and each image region to be detected, as well as the matching degree between the negative prompt text and each image region to be detected.

[0174] The device further includes a determining module, used to determine, from each image region to be detected, an image region whose matching degree with the negative prompt text is no greater than a preset threshold, as a candidate image region.

[0175] In the object detection method provided by this invention, the location prediction model only needs to be trained on the first sample image and the real locations of each object to be detected within it, and the image-text matching model only needs to be trained on the second sample image and its real descriptive text. The training data required for both the location prediction and image-text matching models are available in open-source datasets. When training these two models, the open-source datasets can be directly reused, eliminating the need for manpower to label the location and descriptive text of each object in a variety of images. Furthermore, the location prediction model focuses on optimizing the location prediction loss during training, while the image-text matching model focuses on optimizing the image-text matching loss. Each model has a single and clear training objective, resulting in faster convergence speed, reduced training iterations, and lower hardware costs during training. This invention, by separating object localization and object classification into two independent models, trains the location prediction model and image-text matching model with lower manpower costs and hardware performance requirements to achieve open word detection, reducing the usage and development costs of open word detection.

[0176] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0177] Memory 803 is used to store computer programs;

[0178] The processor 801, when executing the program stored in the memory 803, implements the object detection method described in the foregoing embodiments. The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0179] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0180] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0181] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0182] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described object detection methods.

[0183] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the object detection methods described above.

[0184] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0185] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0186] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, storage medium embodiments, and product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0187] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method of object detection, characterized by, The method comprises: acquiring an image to be detected and a description text to be matched; determining an image region occupied by each to-be-detected object in the to-be-detected image as a to-be-detected image region by using a pre-trained position prediction model, wherein the position prediction model is obtained by training a first sample image and real positions of sample objects in the first sample image; inputting each to-be-detected image region and the description text into a pre-trained image-text matching model to obtain a matching degree of the description text and each to-be-detected image region, wherein the image-text matching model is obtained by training a second sample image and a real description text of the second sample image; selecting an image region with the largest matching degree from the description text from among candidate image regions as an image region matched with the description text in the to-be-detected image, wherein the candidate image regions are determined from the to-be-detected image regions.

2. The method of claim 1, wherein, The position prediction model is trained by the following steps: acquiring a first sample image and real positions of sample objects in the first sample image; inputting the first sample image into an initial structure of the position prediction model to obtain predicted positions of the objects in the first sample image; calculating a loss value based on a difference between the obtained predicted positions and the real positions; adjusting network parameters of the initial structure of the position prediction model based on the loss value until a first convergence condition is reached to obtain the position prediction model.

3. The method of claim 2, wherein, The predicted positions are multiple; The loss value is calculated based on a difference between the obtained predicted positions and the real positions, comprising: for each first sample image, if there are multiple sample objects in the first sample image, for each sample object in the first sample image, calculating an intersection over union of the real position of the sample object and each predicted position of the first sample image; determining a predicted position with the largest intersection over union of the real position of the sample object as a to-be-utilized position corresponding to the sample object; calculating a loss value based on a difference between the real position of each sample object and the corresponding to-be-utilized position.

4. The method according to any one of claims 1 to 3, characterized in that, The image-text matching model is trained by the following steps: acquiring a second sample image and a real description text of the second sample image; inputting the second sample image and the real description text of the second sample image into an initial structure of the image-text matching model to obtain a matching degree of the second sample image and the real description text of the second sample image; adjusting network parameters of the initial structure of the image-text matching model based on the matching degree of the second sample image and the real description text of the second sample image until a second convergence condition is reached to obtain the image-text matching model.

5. The method according to any one of claims 1 to 3, characterized in that, The to-be-detected image regions and the description text are inputted into the pre-trained image-text matching model to obtain a matching degree of the description text and each to-be-detected image region, comprising: input the negative prompt text, the image regions to be detected, and the description text into a pre-trained image-text matching model to obtain a matching degree of the description text and each image region to be detected, and a matching degree of the negative prompt text and each image region to be detected; before the selecting, from the candidate image regions, the image region with the largest matching degree with the description text, the method further includes: from the image regions to be detected, determining an image region with a matching degree with the negative prompt text not greater than a preset threshold as a candidate image region.

6. An object detection device, characterized by, The apparatus includes: a first obtaining module configured to obtain an image to be detected and a description text to be matched; a utilizing module configured to determine, by using a pre-trained position prediction model, an image region occupied by each object to be detected in the image to be detected as an image region to be detected, wherein the position prediction model is obtained by training a first sample image and a real position of a sample object in the first sample image; an input module configured to input the image regions to be detected and the description text into a pre-trained image-text matching model to obtain a matching degree of the description text and each image region to be detected, wherein the image-text matching model is obtained by training a second sample image and a real description text of the second sample image; a selecting module configured to select, from the candidate image regions, an image region with the largest matching degree with the description text as an image region in the image to be detected that matches the description text, wherein the candidate image regions are determined from the image regions to be detected.

7. The apparatus of claim 6, wherein, The position prediction model is trained by the following steps: obtaining a first sample image and real positions of sample objects in the first sample image; inputting the first sample image into an initial structure of the position prediction model to obtain predicted positions of the objects in the first sample image; calculating a loss value based on a difference between the obtained predicted positions and the real positions; adjusting network parameters of the initial structure of the position prediction model based on the loss value until a first convergence condition is reached to obtain the position prediction model; alternatively, the predicted positions are multiple; the calculating of the loss value based on the difference between the obtained predicted positions and the real positions includes: for each first sample image, if there are multiple sample objects in the first sample image, for each sample object in the first sample image, calculating an intersection over union of the real position of the sample object and each predicted position of the first sample image; determining a predicted position with the largest intersection over union of the real position of the sample object as a to-be-utilized position corresponding to the sample object; calculating a loss value based on a difference between the real positions of the sample objects and the corresponding to-be-utilized positions; and / or, the image-text matching model is trained by the following steps: obtaining a second sample image and a real description text of the second sample image; inputting the second sample image and the real description text of the second sample image into an initial structure of a picture-text matching model to obtain a matching degree of the second sample image and the real description text of the second sample image; adjusting network parameters of the initial structure of the picture-text matching model based on the matching degree of the second sample image and the real description text of the second sample image until a second convergence condition is reached to obtain the picture-text matching model; and / or, the input module is specifically configured to input the negative prompt text, each to-be-detected image region, and the description text into a pre-trained picture-text matching model to obtain a matching degree of the description text and each to-be-detected image region and a matching degree of the negative prompt text and each to-be-detected image region; The device further includes: The determination module is configured to determine, from the to-be-detected image regions, an image region having a matching degree with the negative prompt text that is not greater than a preset threshold as a candidate image region.

8. An electronic device, comprising: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is used to store a computer program. The processor is used to execute the program stored on the memory to implement the method of any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-5.

10. A computer program product, characterised in that, comprising a computer program, which, when executed by the processor, implements the method according to any one of claims 1-5. comprising a computer program, which, when executed by the processor, implements the method according to any one of claims 1-5.

Images