Object category determination method and apparatus, storage medium, and electronic device

Abstract

This application discloses a method, apparatus, storage medium, and electronic device for determining object categories. The method includes: extracting initial visual features from candidate regions of an image to be identified; determining the current object category corresponding to the candidate regions based on the similarity between historical text features corresponding to historical object categories and the initial visual features; if the current object category meets the target category conditions, performing a clustering operation on the candidate regions to obtain feature clusters; determining target visual features based on the initial visual features corresponding to the feature clusters; and determining target text features based on the text description information corresponding to the feature clusters; and determining whether the current object category is a newly appearing object category based on the target text features, target visual features, and historical visual features corresponding to historical object categories. This solves the technical problem of low efficiency in existing object detection technologies that rely on large amounts of labeled data for new categories.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a method and apparatus for determining object categories, a storage medium, and an electronic device. Background Technology

[0002] In the field of object detection technology, traditional solutions typically employ a data-driven approach, requiring the collection of large amounts of newly labeled data and model fine-tuning for new categories. However, this approach is highly dependent on data, making the acquisition of sufficient and high-quality labeled samples costly and time-consuming. Furthermore, it is prone to overfitting or underfitting when faced with scarce samples or insufficient intra-class diversity, resulting in low detection accuracy. Therefore, existing methods struggle to quickly adapt to environmental changes or the addition of new categories, often requiring complex meta-sampling strategies or full-parameter retraining. This not only incurs significant resource investment but also fails to effectively address the problem of efficient identification of new targets in open scenarios.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method and apparatus for determining object categories, a storage medium, and an electronic device, to at least solve the technical problem in existing target detection technologies that rely on a large amount of labeled data for new categories, resulting in low efficiency.

[0005] According to one aspect of the embodiments of this application, a method for determining an object category is provided, comprising: extracting initial visual features from candidate regions of an image to be identified; determining a current object category corresponding to the candidate regions based on the similarity between historical text features corresponding to historical object categories and the initial visual features; if the current object category satisfies a target category condition, performing a clustering operation on the candidate regions to obtain feature clusters, determining target visual features based on the initial visual features corresponding to the feature clusters, and determining target text features based on text description information corresponding to the feature clusters; and determining whether the current object category is a newly appearing object category based on the target text features, the target visual features, and the historical visual features corresponding to the historical object categories.

[0006] According to another aspect of the embodiments of this application, an object category determination apparatus is also provided, comprising: an extraction module, configured to extract initial visual features from candidate regions of an image to be identified; a first determination module, configured to determine the current object category corresponding to the candidate region based on the similarity between historical text features corresponding to historical object categories and the initial visual features; a clustering module, configured to perform a clustering operation on the candidate region to obtain feature clusters when the current object category meets the target category conditions, determine target visual features based on the initial visual features corresponding to the feature clusters, and determine target text features based on the text description information corresponding to the feature clusters; and a second determination module, configured to determine whether the current object category is a newly appearing object category based on the target text features, the target visual features, and the historical visual features corresponding to the historical object categories.

[0007] In an exemplary embodiment, before performing clustering on the candidate region to obtain a feature cluster when the current object category meets the target category condition, the device is further configured to: calculate the cosine similarity between the initial visual features and the historical text features to obtain a similarity distribution vector; determine the predicted information entropy based on the similarity distribution vector; determine that the current object category meets the target category condition when the predicted information entropy is greater than or equal to a preset entropy threshold; and determine that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold.

[0008] In an exemplary embodiment, the apparatus is configured to determine that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold by: obtaining the maximum similarity value in the similarity distribution vector; determining the current object category as the historical object category when the maximum similarity value is greater than or equal to a preset similarity threshold; and determining the current object category as the background region category when the maximum similarity value is less than or equal to the similarity threshold.

[0009] In an exemplary embodiment, the apparatus is configured to determine whether the current object category is a first-time occurrence of an object category by: determining a first similarity parameter based on the target text features, the target visual features, and the historical visual features corresponding to the historical object category; determining a second similarity parameter based on the historical visual features and the target visual features; and determining that the current object category is a first-time occurrence of an object category when the first similarity parameter is greater than a first threshold and the second similarity parameter is less than a second threshold.

[0010] In an exemplary embodiment, the apparatus is configured to determine target text features based on text description information corresponding to the feature clusters by: generating a target visual prototype according to the initial visual features corresponding to the feature clusters; inputting the target visual prototype into a pre-trained large language model to generate the text description information; and encoding the text description information to obtain the target text features.

[0011] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described method for determining the object category when it is run.

[0012] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for determining the object category as described above.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described method for determining the object category through the computer program.

[0014] In this embodiment, initial visual features are extracted from candidate regions of the image to be identified; the current object category corresponding to the candidate region is determined based on the similarity between historical text features corresponding to historical object categories and the initial visual features; if the current object category meets the target category conditions, clustering is performed on the candidate region to obtain feature clusters; target visual features are determined based on the initial visual features corresponding to the feature clusters, and target text features are determined based on the text description information corresponding to the feature clusters; based on the target text features, target visual features, and historical visual features corresponding to historical object categories, it is determined whether the current object category is the first object category to appear. By combining target visual features and historical visual features to perform cross-modal consistency verification and category difference isolation judgment, new categories are automatically confirmed. This achieves the technical effect of accurately identifying and registering unknown newly emerging target object categories without manual pre-annotation, while effectively filtering background interference. This solves the technical problem of low efficiency in existing object detection technologies that rely on a large amount of labeled data for new categories. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 This is a schematic diagram of an application environment for an optional method for determining object categories according to an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating an optional method for determining an object category according to an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of an optional method for determining an object category according to an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the structure of an optional object category determination device according to an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] The present application will be described below with reference to embodiments:

[0024] According to one aspect of the embodiments of this application, a method for determining an object category is provided.

[0025] Optionally, in this embodiment, the above-mentioned method for determining object categories can be applied to the fields of intelligent transportation, video surveillance, autonomous driving, and intelligent manufacturing, and to scenarios involving real-time detection and classification of new targets (such as traffic cones, parking signs, and abnormal objects). Specifically, it can be applied to scenarios such as... Figure 1 The method for determining the above-mentioned object categories is applied in the hardware environment consisting of server 101 and terminal device 103.

[0026] For example, when applying the above object category determination method in a parking lot scenario, the new target may include, but is not limited to, traffic cones, parking signs, and trolleys; when applying the above object category determination method in an autonomous driving scenario, the new target may include, but is not limited to, animals, construction machinery, construction signs, construction roadblocks, fallen trees, scattered large goods, abnormal traffic barriers, and roadblocks.

[0027] Furthermore, combined Figure 1 As shown, the method for determining the above-mentioned object category can be implemented by the terminal device 103 or the server 101 respectively, or by the terminal device 103 and the server 101 together. This application embodiment does not limit this.

[0028] It should be noted that the aforementioned server 101 is connected to the terminal device 103 via a network and can be used to provide services to the terminal device or the application 107 installed on the terminal device to implement the aforementioned method for determining object categories. A database 105 can also be set up on or independently of the server 101 to provide data storage services for implementing the aforementioned method for determining object categories. Specifically:

[0029] The aforementioned server 101 and terminal device 103 can be any node in a distributed system, such as a blockchain system. This blockchain system can be formed by connecting multiple nodes through network communication. The nodes can form any type of network, and any type of computing device, such as any electronic device, can become a node in this distributed system by joining the network formed between the nodes.

[0030] The aforementioned server 101 can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0031] The aforementioned networks may include, but are not limited to, wired networks and wireless networks. The wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication.

[0032] The aforementioned terminal device 103 may be a terminal configured with an application, and may include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, virtual reality (VR) terminals, augmented reality (AR) terminals, mixed reality (MR) terminals, and other computer devices.

[0033] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the methods for determining the above object categories include:

[0034] S202, Extract initial visual features from candidate regions of the image to be identified;

[0035] Optionally, in the embodiments of this application, the aforementioned candidate region can be understood as a region of interest in the image that may contain the target object, specifically:

[0036] In scenarios where candidate regions are generated based on region proposal networks, this includes, but is not limited to, anchor boxes generated by any neural network, regions generated by sliding windows, or superpixel regions generated by segmentation algorithms.

[0037] Optionally, in the embodiments of this application, the aforementioned initial visual features may be vector representations reflecting local semantic information of an image, including but not limited to high-dimensional feature vectors extracted by a convolutional neural network and feature embeddings after normalization.

[0038] It should be noted that this application does not limit the generation method of the above candidate regions, the specific network structure for extracting the above initial visual features, or the feature dimensions.

[0039] For example, in an intelligent traffic monitoring scenario, the candidate region can be a rectangular box in the image that is suspected to be a vehicle, pedestrian or traffic cone, and the initial visual features can be a feature vector extracted by the ResNet backbone and then subjected to global average pooling.

[0040] S204, determine the current object category corresponding to the candidate region based on the similarity between the historical text features and the initial visual features corresponding to the historical object category;

[0041] Optionally, in the embodiments of this application, the above-mentioned historical text features can be understood as transforming the semantic description of a known category into a representation in a vector space. Specifically, in detection scenarios based on pre-trained multimodal models, this includes, but is not limited to, embedding vectors generated by directly inputting category names and embedding vectors generated by manually designed descriptive statements.

[0042] Optionally, in another application scenario, such as industrial quality inspection, the aforementioned historical text features can also be a semantic representation of product defect types in this embodiment of the application.

[0043] It should be noted that this application does not limit the specific sources of the aforementioned historical text features, the extraction methods of the aforementioned initial visual features, or the mathematical formulas for similarity calculation.

[0044] For example, in a traffic monitoring scenario, the aforementioned historical text features can be vectors of standard category names such as "cones" and "parking signs" processed by a text encoder, while the aforementioned initial visual features are vectors of suspected object regions cropped from an image processed by a visual encoder.

[0045] S206, if the current object category meets the target category condition, perform clustering operation on the candidate region to obtain feature clusters, determine the target visual features based on the initial visual features corresponding to the feature clusters, and determine the target text features based on the text description information corresponding to the feature clusters;

[0046] Optionally, in the embodiments of this application, the above-mentioned feature clusters can be understood as a set of candidate regions with similar visual semantics. Specifically, in the scenario based on density clustering, it includes, but is not limited to, dense point groups formed by the DBSCAN algorithm, high-density regions divided by the K-Means algorithm, and similar node groups merged by hierarchical clustering.

[0047] Optionally, in the embodiments of this application, the above-mentioned target visual features may be representative vectors that reflect the overall semantics of the cluster, including but not limited to the weighted mean vector of all samples in the cluster, the median vector in the cluster, or the sample vector closest to the center point.

[0048] It should be noted that this application does not limit the specific type of clustering algorithm, the calculation method of the target visual features, or the source of the text description information.

[0049] For example, in a traffic monitoring scenario, the aforementioned feature cluster can be a clustering group formed by extracting visual features from multiple candidate area boxes of suspected reflective warning frames. The aforementioned target visual feature is a weighted average vector of all visual features within the cluster, and the aforementioned target text feature is a natural language description vector generated by a large language model based on the visual feature.

[0050] S208. Based on the target text features, target visual features, and historical visual features corresponding to historical object categories, determine whether the current object category is the first object category to appear.

[0051] Optionally, in the embodiments of this application, the historical visual features corresponding to the above-mentioned historical object categories can be understood as representative visual vectors of registered known categories. Specifically, in the scenario based on prototype learning, this includes, but is not limited to, the visual prototype vectors of registered categories, the mean vector of the support set samples of registered categories, and the centroid vector of the feature space of registered categories.

[0052] Optionally, in the embodiments of this application, the above-mentioned target visual features may be an aggregated vector that reflects the overall semantics of the newly discovered target, including but not limited to the weighted feature vector of the cluster center, the first principal component vector after principal component analysis, or the feature vector of the most typical sample in the cluster.

[0053] It should be noted that this application does not limit the specific storage format of the historical visual features corresponding to the aforementioned historical object categories, the specific judgment logic for determining the first appearance of an object category (such as threshold comparison, distance measurement, etc.), or the interaction method between the aforementioned target text features and historical features.

[0054] For example, in an intelligent transportation scenario, the historical visual features corresponding to the aforementioned historical object categories can be the visual prototypes of known categories such as "traffic cones" and "cars". The aforementioned target visual features are the visual center vectors of the newly discovered "folding warning rack" cluster. By calculating the similarity between the target visual features and the historical visual features of all known categories, if all are lower than a preset threshold, it is determined to be the object category that appears for the first time.

[0055] In an exemplary embodiment, the above-described method for determining object categories can be applied to the field of image processing technology. Considering that in different real-world application scenarios, when new types of targets need to be detected, traditional deep learning solutions typically face the following challenges: strong data dependence, high data preparation costs, long development cycles, and difficulty in quickly adapting to changes in target types or environments for specific scenarios. This data-driven development model often leads to extended project implementation cycles and increased resource investment, especially when facing urgent needs or small sample scenarios, resulting in low efficiency. Specifically:

[0056] The object category determination method described above enables an open-vocabulary object detection and continuous learning system based on CLIP. Through multimodal semantic understanding and a dynamic evolution mechanism, it achieves efficient object detection under conditions of few or even zero samples. The object category determination method utilizes OpenAI pre-trained weights for rapid startup, combined with a dynamic adapter for efficient fine-tuning, and leverages an uncertainty mechanism to reduce manual workload, ultimately forming a closed-loop learning system. This effectively solves key bottlenecks of traditional methods in areas such as sample dependence, category expansion, and environmental adaptation, providing a feasible technical path for the intelligent upgrading of fields such as intelligent transportation, intelligent manufacturing, medical diagnosis, and ecological monitoring.

[0057] First, acquire the image to be identified and extract candidate regions containing potential targets;

[0058] A pre-trained cross-modal encoder is used to compare the initial visual features of the candidate region with the historical text features of the known category to obtain the predicted information entropy, and then the current object category is determined based on the predicted information entropy.

[0059] When the predicted information entropy is greater than the preset entropy threshold, it is determined that the current object category meets the target category condition, and the candidate region is included in the clustering process to form a feature cluster with semantic consistency.

[0060] Target visual features are generated by aggregating visual data within clusters, and target text features are generated by combining semantic descriptions generated by large language models.

[0061] The feature representation of the new category is compared and analyzed with the visual features of historical object categories stored in the database. If the similarity is lower than a set threshold, the category is determined to be a new target appearing for the first time, thus completing the closed-loop processing flow from local area detection to global category discovery.

[0062] This application's embodiments extract initial visual features from candidate regions of an image to be identified; determine the current object category corresponding to the candidate region based on the similarity between historical text features corresponding to historical object categories and the initial visual features; if the current object category meets the target category conditions, perform clustering operations on the candidate region to obtain feature clusters; determine the target visual features based on the initial visual features corresponding to the feature clusters, and determine the target text features based on the text description information corresponding to the feature clusters; based on the target text features, target visual features, and historical visual features corresponding to historical object categories, determine whether the current object category is a newly appearing object category; by combining the target visual features and historical visual features for cross-modal consistency verification and category difference isolation judgment, new categories are automatically confirmed, achieving the technical effect of accurately identifying and registering unknown newly appearing target object categories without manual pre-annotation, while effectively filtering background interference. Under conditions of scarce samples, variable categories, and complex environments, it achieves high-precision, high-generalization, and continuously evolving target detection capabilities, thereby solving the technical problem of low efficiency in existing target detection technologies that rely on a large amount of labeled data for new categories.

[0063] As an optional approach, before performing clustering operations on the candidate regions to obtain feature clusters when the current object category meets the target category conditions, the method further includes: calculating the cosine similarity between the initial visual features and the historical text features to obtain a similarity distribution vector; determining the predicted information entropy based on the similarity distribution vector; determining that the current object category meets the target category conditions when the predicted information entropy is greater than or equal to a preset entropy threshold; and determining that the current object category does not meet the target category conditions when the predicted information entropy is less than the entropy threshold.

[0064] Optionally, in the embodiments of this application, the aforementioned similarity distribution vector refers to a set of values ​​reflecting the probability of a candidate region matching multiple known categories, including but not limited to the cosine similarity score sequence after normalization, the probability distribution vector obtained by transformation through the Softmax activation function, and the log probability vector after temperature coefficient scaling adjustment.

[0065] Optionally, in the embodiments of this application, the aforementioned prediction information entropy refers to a quantitative indicator that measures the uncertainty of the model prediction results, including but not limited to the Shannon entropy value calculated based on the negative log probability, the uncertainty measure derived based on the maximum entropy principle, and a statistical measure that reflects the degree of dispersion of classification confidence.

[0066] It should be noted that this application does not limit the specific dimensions of the cosine similarity calculation, the specific mathematical formula of the predicted information entropy, or the specific basis for setting the entropy threshold.

[0067] For example, in a traffic monitoring scenario, the visual features of the candidate region and the text features of ten known traffic signs are used to calculate ten similarity scores. After normalization, a probability distribution vector containing ten elements is formed, and then the predicted information entropy is calculated using this distribution vector.

[0068] For example, if the calculated predicted information entropy is 0.85 and the preset entropy threshold is 0.80, the system determines that the current object category meets the target category condition and triggers the subsequent new category confirmation process; if the calculated predicted information entropy is 0.30 and the preset entropy threshold is 0.40, the system determines that the current object category does not meet the target category condition, marks the area as background or a known category, and discards it.

[0069] Through the embodiments of this application, the candidate region screening based on uncertainty measurement is adopted, which effectively reduces redundant calculations and the burden of manual review, thereby achieving the goal of improving the overall operating efficiency and accuracy of the new target detection system.

[0070] As an optional approach, determining that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold includes: obtaining the maximum similarity value in the similarity distribution vector; determining the current object category as the historical object category when the maximum similarity value is greater than or equal to a preset similarity threshold; and determining the current object category as the background region category when the maximum similarity value is less than or equal to the similarity threshold.

[0071] Optionally, in the embodiments of this application, the maximum similarity value mentioned above refers to the element with the largest value in the similarity distribution vector, that is, the matching score of the candidate region with the category most similar to all known categories, including but not limited to the scalar value obtained by directly taking the maximum value operation, the maximum value after smoothing, and the single-point metric value reflecting the highest confidence.

[0072] Optionally, in the embodiments of this application, the aforementioned preset similarity threshold refers to the classification boundary used to distinguish between high-confidence matches and low-confidence matches, including but not limited to fixed values ​​set by experience, critical values ​​dynamically calculated based on the statistical distribution of the validation set, and fixed percentage thresholds set according to business needs.

[0073] Optionally, in the embodiments of this application, the aforementioned historical object category refers to a known target type that has been registered in the system and has a complete feature description, including but not limited to standard categories accumulated through historical iterations, benchmark categories confirmed by humans, and established categories that have corresponding visual prototypes and text descriptions in the database.

[0074] Optionally, in the embodiments of this application, the aforementioned background region category refers to a non-target region that does not belong to any known target type, including but not limited to pure background pixel regions, interference noise regions, and other undefined irrelevant object regions.

[0075] It should be noted that this application does not limit the specific extraction method of the maximum similarity value, the specific numerical setting logic of the preset similarity threshold, the specific definition range of the historical object category, or the specific division criteria of the background area category.

[0076] Through the embodiments of this application, a dual judgment process based on the maximum similarity threshold is adopted to achieve the technical effect of accurately distinguishing known targets from background interference, thereby reducing the false detection rate and improving the robustness of the target detection system.

[0077] As an optional approach, determining whether the current object category is a newly appearing object category based on the target text features, the target visual features, and the historical visual features corresponding to the historical object categories includes: determining a first similarity parameter based on the target text features and the target visual features; determining a second similarity parameter based on the historical visual features and the target visual features; and determining that the current object category is a newly appearing object category when the first similarity parameter is greater than a first threshold and the second similarity parameter is less than a second threshold.

[0078] Optionally, in the embodiments of this application, the historical visual features corresponding to the aforementioned historical object categories refer to a representative set of known target types that have been registered and verified in the system, including but not limited to known category center prototype vectors generated by clustering, typical sample embedding vectors stored in the feature library, and category mean vectors calculated based on statistical distribution.

[0079] Optionally, in the embodiments of this application, the aforementioned first similarity parameter refers to a quantitative indicator that measures the degree of semantic alignment between target text features and target visual features, including but not limited to calculating the cosine similarity value of the angle between their vectors, calculating the reciprocal of the Euclidean distance between the vectors, and the joint probability score that reflects the confidence of cross-modal matching.

[0080] Optionally, in the embodiments of this application, the aforementioned second similarity parameter refers to a quantitative indicator that measures the degree of appearance similarity between the target visual feature and the known historical visual features, including but not limited to the maximum similarity score between the target visual feature and all historical visual features, the distance metric between the target visual feature and the closest historical prototype, and the lower limit of the confidence interval reflecting the distinguishability between the new target and the old category.

[0081] Optionally, in the embodiments of this application, the first threshold mentioned above refers to the minimum standard used to ensure consistency between the semantic description and visual representation of the new category, including but not limited to a fixed value set empirically, a dynamic threshold based on validation set performance optimization, and a critical probability value reflecting the quality of cross-modal alignment.

[0082] Optionally, in the embodiments of this application, the aforementioned second threshold refers to the highest standard used to ensure that the new category is significantly different from the known category, including but not limited to a fixed value set empirically, a dynamic threshold calculated based on the feature space distribution, and a critical distance value reflecting the category isolation degree.

[0083] It should be noted that this application does not limit the specific generation model of the target text features, the specific extraction network of the target visual features, the specific storage structure of the historical visual features, the specific calculation formula of the first similarity parameter and the second similarity parameter, or the specific setting strategy of the first threshold and the second threshold.

[0084] In an exemplary embodiment, a pre-trained large language model is first used to generate descriptive text based on the visual features of candidate regions. This text is then converted into target text features by a text encoder. Simultaneously, target visual features are extracted by a visual encoder. The cosine similarity between the two is calculated as a first similarity parameter to verify semantic consistency. Subsequently, the target visual features are compared with the historical visual features of all historical object categories in the database. The minimum distance or maximum similarity is calculated as a second similarity parameter to verify novelty. When the first similarity parameter is higher than 0.75 and the second similarity parameter is lower than 0.30, the current object category is determined to be the first object category to appear, thus confirming it as a valid new category.

[0085] Understandably, the pre-trained large language model here can be an autoregressive language model or a large language model based on the Transformer architecture. Specifically, it includes a self-attention module, a feedforward neural network module, a normalization module, and a positional encoding module. The self-attention module is used to capture the global dependencies between different words in the text sequence and calculate attention weights. The feedforward neural network module is used to perform non-linear transformations and feature enhancements on the features at each position. The normalization module is used to stabilize the training process and accelerate model convergence. The positional encoding module is used to provide the model with information on the relative or absolute position of words in the sequence to maintain the word order logic.

[0086] Through the embodiments of this application, a dual judgment process based on cross-modal consistency verification and historical category difference comparison is adopted to achieve the technical effect of accurately identifying and registering new target categories.

[0087] As an optional approach, the above-mentioned determination of target text features based on the text description information corresponding to the feature clusters includes: generating a target visual prototype based on the initial visual features corresponding to the feature clusters; inputting the target visual prototype into a pre-trained large language model to generate the text description information; and encoding the text description information to obtain the target text features.

[0088] Optionally, in the embodiments of this application, the aforementioned target visual prototype refers to a representative feature vector calculated by aggregating the visual features of multiple candidate regions within a feature cluster, including but not limited to the mean vector obtained by weighted averaging of visual features within the cluster, the center vector determined based on density clustering core points, and the statistical feature vector reflecting the main semantic information within the cluster.

[0089] Optionally, in the embodiments of this application, the above-mentioned text description information refers to natural language text generated by the large language model based on the visual prototype for describing the semantics of the target category, including but not limited to complete sentences containing the target subject, attributes, material, color, shape and scene context, standardized descriptive statements generated by structured constraints, and composite text fragments that integrate multimodal information.

[0090] Optionally, in the embodiments of this application, the aforementioned target text features refer to the vector representation obtained after encoding the generated text description information, which is in the same semantic space as the visual features, including but not limited to the sentence-level embedding vector output by the text encoder, the fixed-length vector obtained after pooling operation, and the high-dimensional feature representation that reflects the semantics of the text.

[0091] It should be noted that this application does not limit the specific aggregation algorithm of the above-mentioned target visual prototype, the specific architecture of the above-mentioned pre-trained large language model, the generation strategy of the above-mentioned text description information, or the type of text encoder used in the above-mentioned encoding process.

[0092] For example, in an intelligent transportation scenario, the feature cluster contains cone image features from multiple different perspectives. The system calculates the weighted average of these features to obtain the target visual prototype, which is then input into a large language model to generate the descriptive text "orange cone-shaped traffic barrier". The text is then converted into target text features by a text encoder.

[0093] For example, in an industrial quality inspection scenario, the samples within a feature cluster are all metal parts with specific scratches. The system generates a target visual prototype and inputs it into a large language model. The model generates the descriptive text "metal disk with linear scratches on the surface". The text is then encoded to obtain the target text features, which are used for subsequent new category registration.

[0094] For example, in a warehousing and logistics scenario, if the feature cluster corresponds to stacked cardboard boxes, the target visual prototype generated by the system is input into a large language model, and the model generates descriptive text "stacked brown corrugated cardboard boxes". The encoded target text features will be used to calculate similarity with known category features.

[0095] In an exemplary embodiment, firstly, a weighted average is calculated on the visual features of all candidate regions within a feature cluster to generate a target visual prototype that can represent the overall semantics of the cluster. Then, the visual prototype is sent as an input prompt to a pre-trained large language model. The large language model combines its internal knowledge base to generate textual description information such as "folding traffic warning rack with red and white reflective strips". Finally, the text encoder is used to convert the description information into high-dimensional target text features so as to perform cross-modal comparison with the visual features.

[0096] Through the embodiments of this application, cross-modal mapping based on visual prototype generation and text encoding is achieved, realizing the technical effect of transforming visual clustering results into standardized semantic descriptions, thereby enhancing the model's ability to understand the semantics of new categories and improving the accuracy of subsequent detection.

[0097] In one exemplary embodiment, such as Figure 3 As shown, the above method for determining object categories can be applied to the transportation field, specifically:

[0098] Cross-modal pre-training: CLIP dual encoders are pre-trained by jointly training the visual and text encoders on large-scale image-text pairs. The goal is to establish a cross-modal shared embedding space, making matching image-text pairs feature-similar and mismatched pairs feature-separated. Initially, pre-trained weights released by OpenAI are directly loaded, eliminating the need for retraining from scratch. The contrastive learning task in the pre-training phase is unrelated to the downstream detection task, but the semantic alignment ability learned by the encoder can be transferred to new tasks (such as zero-shot classification or detection).

[0099] Visual-text joint proposal: The input image is processed by a CLIP visual encoder to extract global features, and a lightweight RPN network generates candidate region proposals (RoIs). This model employs a two-branch architecture for cross-modal matching. The visual processing branch crops each RoI and extracts its features using the CLIP visual encoder. (Based on the initial visual features mentioned above), the text processing branch uses the CLIP text encoder to convert the input text description into an embedded representation. (The aforementioned historical text features). Finally, cross-modal matching is achieved by calculating the similarity between visual features and text embeddings. The similarity calculation formula is as follows:

[0100]

[0101]

[0102] Where N is the number of RoIs, K is the number of categories, i represents the i-th visual candidate region ROI, and j represents the j-th text category or text description;

[0103] Uncertainty-based candidate filtering: similarity distribution for each RoI Calculate entropy:

[0104]

[0105] Based on the entropy value of each RoI region and maximum similarity Based on the relationship, RoI regions can be divided into three categories: new category, background, and known category, where, for the i-th candidate region, the similarity vector Si = [s] between it and the text embeddings of all K known categories is used to classify RoI regions into three categories: new category, background, and known category. i1 s i2 …s iK Find the maximum similarity in [the context]. This should be the maximum similarity. This represents the degree to which the region matches the best-matching known category text. As shown in Table 1, this is based on an entropy threshold. and similarity threshold RoI region classification processing instructions:

[0106] Table 1

[0107]

[0108] S4, Dynamic New Category Confirmation: First, the CLIP visual encoder extracts candidate region features for density clustering (DBSCAN, ε=0.35), forming semantically similar candidate clusters. Next, the weighted visual prototype (Gaussian weighted mean, σ=0.2) of each cluster is calculated. Then, the CLIP text encoder maps the prototypes to the text space, driving the LLM to generate a canonical description (the aforementioned target text features). Specifically, using CLIP's shared space, the most relevant text fragments (as anchors) are retrieved for the visual prototype pk. These text anchors are then used as context, and the Prompt project drives the LLM, which outputs a canonical text description (e.g., category name or precisely defined vocabulary). This description serves as the semantic identifier for the new category. The canonical text description here typically... The initial category name is either a single category name or a short, restrictive phrase. Then, a dual-encoder joint verification is performed, requiring a visual-text similarity cos(p_k,t_k) > 0.7 and a maximum similarity to known categories < 0.3. Finally, registration is completed by deduplication in the feature space (||p_k - p_j^{known}||_2 > 0.5). For example, the L2 distance between a newly discovered visual prototype and all registered category prototypes is calculated. If the minimum distance is greater than a threshold (e.g., 0.5), it is determined to be a completely new category and registered (the prototype and description are stored); otherwise, it is considered a duplicate or variant of a known category and registration is rejected. The L2 distance, or Euclidean distance, measures the geometric linear distance between two vectors (here, the visual prototype vector) in the multidimensional feature space. The entire process achieves precise alignment of visual and semantic information through CLIP's shared embedding space.

[0109] Understandably, in autonomous driving scenarios, the dynamic new category confirmation process may discover:

[0110] When the CLIP visual encoder extracts a set of unlabeled object features (such as a tripod-shaped object with reflective strips), it generates a visual prototype after density clustering.

[0111] LLM generates a description based on the scene context: "A foldable warning frame for road construction, with red and white reflective stripes";

[0112] Dual encoder verification confirmed that the description had a cosine similarity of 0.82 with the visual prototype and a similarity of only 0.25 with the known category "traffic cone".

[0113] Ultimately, this new category was registered as the Foldable Reflective Warning Stand category, with the following typical characteristics added: unfolded width 1.2-1.5 meters, height 0.6-0.8 meters, and reflective strip spacing 15cm.

[0114] Understandably, when a system continuously learns new categories, human review is necessary to eliminate safety risks caused by algorithmic uncertainty and ensure the reliability of model iteration. For example, in the new category generation process of an autonomous driving system, human review ensures reliability through multi-node intervention: when the confidence score of the LLM-generated description is below a threshold (e.g., <0.6), the cross-modal similarity is in the fuzzy range (0.4-0.7), or a high-risk object is detected, the system automatically triggers human review. The high-risk object identification process here includes, but is not limited to:

[0115] First, high-risk objects are defined based on their attributes and categories, such as pedestrians, cyclists, and vulnerable groups. Compared to motor vehicles, these objects have extremely low tolerance for collisions, and their behavior is difficult to predict. The abnormal state (such as damage or displacement) of critical public facilities like traffic lights, guardrails, and roadblocks directly affects driving safety. Second, based on the definition of perception confidence and data quality, when the classification confidence of an object is below a set threshold, its category cannot be determined. This uncertainty itself is high-risk and requires manual confirmation to avoid missed or false detections. Partial occlusion of an object leads to the loss of key features (such as license plates and pedestrian gestures), making it difficult to accurately predict its intentions. Finally, the definition can be based on motion state. For example, if an object's speed or direction changes drastically in a short period, there is a high probability of collision with the predicted trajectory of a vehicle. In other words, identifying high-risk objects is the result of a multi-dimensional comprehensive scoring and verification of attributes, confidence, and state.

[0116] At this point, the auditor can make a judgment through the multi-evidence panel (including sensor data, similar category comparison, and an editable description box), and can choose to confirm, reject, or subdivide into new subcategories. Specifically:

[0117] First, the data is graded using confidence filtering and dynamic weighting: manually labeled data is assigned a weight of 1.5, automatically labeled data with high confidence (>0.8) retains its original weight, and data with low confidence is weighted down to 0.5. During training, a progressive parameter unfreezing process is employed: for epochs < 5, only the adapter (e.g., LoRA fine-tuning of the visual encoder) and the Prompt update of the text encoder are tuned; for epochs < 10, only the last two layers of the visual encoder are unfrozen; and for epochs ≥ 10, the last layer of the text encoder is unfrozen. This effectively improves the detection accuracy of new categories while maintaining stable recognition performance of existing categories and significantly shortens the model iteration cycle.

[0118] Through the embodiments of this application, based on the similarity between visual features and text embeddings, and utilizing uncertain candidate filtering rules, redundant information and subsequent manual review are reduced, thereby improving the detection accuracy of small or occluded objects. Simultaneously, a dynamic new category confirmation module is integrated, significantly improving the efficiency and accuracy of open-world object detection through text-visual collaborative verification and incremental knowledge updates.

[0119] Simultaneously, by combining historical manual review with model uncertainty (such as samples with high prediction entropy), intelligent push notifications can be sent for boundary cases requiring manual intervention, reducing review workload by over 70%. Attribute annotations added by reviewers can generate new text descriptions, expanding the semantic expressive power of the text encoder and addressing the original CLIP's insufficient capture of ambiguous concepts. Review intervention results serve as high-confidence labeled data, guiding the small-sample incremental updates of the CLIP dual encoder and avoiding catastrophic forgetting in traditional continuous learning. The dual encoder WeChat employs progressive parameter unfreezing to avoid overfitting caused by full parameter updates; CLIP's cross-modal alignment capability allows the system to directly generate detectors from text input without requiring any new category of labeled data or model fine-tuning, whereas self-supervised methods must rely on a small number of labeled samples for adaptation. It supports the detection of abstract attributes, while self-supervised methods can only recognize visual patterns seen during pre-training and lack semantic generalization ability.

[0120] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0121] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0123] Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0124] According to another aspect of the embodiments of this application, an apparatus for determining object categories for implementing the above-described object category determination method is also provided. This object category determination apparatus can be used to implement the object category determination method provided in the above embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0125] Figure 4 This is a structural block diagram of an optional object category determination device according to an embodiment of this application, such as... Figure 4 As shown, the device for determining the object category includes:

[0126] Extraction module 402 is used to extract initial visual features from candidate regions of the image to be identified;

[0127] The first determining module 404 is used to determine the current object category corresponding to the candidate region based on the similarity between the historical text features corresponding to the historical object category and the initial visual features;

[0128] Clustering module 406 is used to perform clustering operation on the candidate region to obtain feature clusters when the current object category meets the target category conditions, determine target visual features based on the initial visual features corresponding to the feature clusters, and determine target text features based on the text description information corresponding to the feature clusters.

[0129] The second determining module 408 is used to determine whether the current object category is the first object category to appear, based on the target text features, the target visual features, and the historical visual features corresponding to the historical object category.

[0130] In an exemplary embodiment, before performing clustering on the candidate region to obtain a feature cluster when the current object category meets the target category condition, the device is further configured to: calculate the cosine similarity between the initial visual features and the historical text features to obtain a similarity distribution vector; determine the predicted information entropy based on the similarity distribution vector; determine that the current object category meets the target category condition when the predicted information entropy is greater than or equal to a preset entropy threshold; and determine that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold.

[0131] In an exemplary embodiment, the apparatus is configured to determine that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold by: obtaining the maximum similarity value in the similarity distribution vector; determining the current object category as the historical object category when the maximum similarity value is greater than or equal to a preset similarity threshold; and determining the current object category as the background region category when the maximum similarity value is less than or equal to the similarity threshold.

[0132] In an exemplary embodiment, the apparatus is configured to determine whether the current object category is a first-time occurrence of an object category by: determining a first similarity parameter based on the target text features, the target visual features, and the historical visual features corresponding to the historical object category; determining a second similarity parameter based on the historical visual features and the target visual features; and determining that the current object category is a first-time occurrence of an object category when the first similarity parameter is greater than a first threshold and the second similarity parameter is less than a second threshold.

[0133] In an exemplary embodiment, the apparatus is configured to determine target text features based on text description information corresponding to the feature clusters by: generating a target visual prototype according to the initial visual features corresponding to the feature clusters; inputting the target visual prototype into a pre-trained large language model to generate the text description information; and encoding the text description information to obtain the target text features.

[0134] Regarding the apparatus in the above embodiments, the terms "module" or "unit" refer to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit. The specific manner in which each module performs its operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

[0135] According to another aspect of the embodiments of this application, an electronic device is provided.

[0136] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps in any of the above method embodiments via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor. Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0137] According to one aspect of this application, a computer program product is also provided, which includes a computer program.

[0138] The computer program product includes a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable media 511. When the computer program is executed by central processing unit 501, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0139] Figure 5 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in ROM 502 or programs loaded into RAM 503 from storage section 508. Random access memory 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0140] The following components are connected to I / O interface 505: input section 506 including keyboard, mouse, etc.; output section 507 including cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; storage section 508 including hard disk, etc.; and communication section 509 including network interface card, modem, etc. Communication section 509 performs communication processing via a network such as the Internet. Drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0141] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0142] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer programs / instructions. For example, embodiments of this application include a computer program / instruction comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions defined in the system of this application. In such embodiments, the computer program / instruction can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program / instruction is executed by a central processing unit, it performs the aforementioned method for determining the object category.

[0143] According to one aspect of this application, a computer-readable storage medium is also provided.

[0144] The processor of the aforementioned electronic device can read the computer instructions from a computer-readable storage medium, and the processor executes the computer instructions, causing the electronic device to perform the object category determination method provided in the various optional implementations of the object category determination aspect.

[0145] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store methods for performing the embodiments of this application.

[0146] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0147] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0148] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining an object category, characterized in that, include: Extract initial visual features from candidate regions of the image to be identified; The current object category corresponding to the candidate region is determined based on the similarity between the historical text features corresponding to the historical object category and the initial visual features. If the current object category meets the target category condition, perform clustering operation on the candidate region to obtain feature clusters, determine the target visual features based on the initial visual features corresponding to the feature clusters, and determine the target text features based on the text description information corresponding to the feature clusters. Based on the target text features, the target visual features, and the historical visual features corresponding to the historical object categories, it is determined whether the current object category is the first object category to appear.

2. The method according to claim 1, characterized in that, Before performing clustering operations on the candidate regions to obtain feature clusters when the current object category meets the target category condition, the method further includes: Calculate the cosine similarity between the initial visual features and the historical text features to obtain a similarity distribution vector; The predicted information entropy is determined based on the similarity distribution vector; If the predicted information entropy is greater than or equal to a preset entropy threshold, the current object category is determined to satisfy the target category condition. If the predicted information entropy is less than the entropy threshold, it is determined that the current object category does not meet the target category condition.

3. The method according to claim 2, characterized in that, The step of determining that the current object category does not meet the target category condition when the predicted information entropy is less than the entropy threshold includes: Obtain the maximum similarity value in the similarity distribution vector; If the maximum similarity value is greater than or equal to a preset similarity threshold, the current object category is determined as the historical object category; If the maximum similarity value is less than or equal to the similarity threshold, the current object category is determined as the background region category.

4. The method according to claim 1, characterized in that, The step of determining whether the current object category is a newly appearing object category based on the target text features, the target visual features, and the historical visual features corresponding to the historical object category includes: A first similarity parameter is determined based on the target text features and the target visual features; A second similarity parameter is determined based on the historical visual features and the target visual features; If the first similarity parameter is greater than the first threshold and the second similarity parameter is less than the second threshold, the current object category is determined to be the object category that appears for the first time.

5. The method according to claim 1, characterized in that, Determining the target text features based on the text description information corresponding to the feature cluster includes: Generate a target visual prototype based on the initial visual features corresponding to the feature cluster; The target visual prototype is input into a pre-trained large language model to generate the text description information; The text description information is encoded to obtain the target text features.

6. A device for determining the category of an object, characterized in that, include: The extraction module is used to extract initial visual features from candidate regions of the image to be identified; The first determining module is used to determine the current object category corresponding to the candidate region based on the similarity between the historical text features corresponding to the historical object category and the initial visual features; The clustering module is used to perform clustering operations on the candidate regions to obtain feature clusters when the current object category meets the target category conditions, determine the target visual features based on the initial visual features corresponding to the feature clusters, and determine the target text features based on the text description information corresponding to the feature clusters. The second determining module is used to determine whether the current object category is the first object category to appear, based on the target text features, the target visual features, and the historical visual features corresponding to the historical object category.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images