Target environment recognition model training method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a target environment recognition model training method and device, electronic equipment and a storage medium. The method comprises: evaluating and processing a plurality of candidate environment label information corresponding to each target environment image set to obtain corresponding label evaluation results; performing label determination processing on the plurality of candidate environment label information to obtain target sub-environment label information corresponding to each target environment image set; performing association processing on the target sub-environment label information to obtain target overall environment label information corresponding to each target environment image set; determining pseudo-labels corresponding to each target environment image set, and adding target environment image sets with pseudo-labels to a preset labeled training set to obtain a target environment image training set; and performing supervised fine-tuning training on a preset multi-modal model based on the target environment image training set to obtain a target environment recognition model. The embodiments of the present application can effectively improve the reliability and comprehensiveness of image environment recognition.

Description

Technical Field

[0001] This application relates to the field of model training technology, and in particular to a method, apparatus, electronic device and storage medium for training a target environment recognition model. Background Technology

[0002] With the continuous accumulation of shop image data on internet platforms such as food delivery and e-commerce, the demand for automatic identification of the shop's environment and attributes based on environmental images from different perspectives, such as the shopfront, lobby, and dining area, is constantly increasing. In practical applications, the environmental characteristics of different shops, such as geographical location and dining conditions, vary greatly. If the shop environment identification is inaccurate, it can easily lead to discrepancies between the shop environment classification results and the actual situation, thus affecting subsequent shop selection, classification, and display. Existing environmental image processing methods mostly rely on manual screening or rule verification, which is not only costly and inefficient, but also difficult to stably and comprehensively process large-scale existing data and continuously added image data, resulting in low comprehensiveness and reliability of environmental image recognition. Summary of the Invention

[0003] This application provides a target environment recognition model training method, apparatus, electronic device, and storage medium, which can improve the comprehensiveness and reliability of image recognition. The above technical solution is as follows: In a first aspect, embodiments of this application provide a method for training a target environment recognition model, including: Obtain multiple target environment image sets corresponding to multiple target objects, wherein each target object in the multiple target objects' target environment image set includes multiple target object environment images obtained from multiple perspectives; By using multiple preset recognition models to perform feature recognition on each of the above target environment image sets, multiple candidate environment label information corresponding to each of the above target environment image sets are obtained. Among them, the multiple recognition models have different model configurations. Based on the preset evaluation dimensions, the multiple candidate environment label information corresponding to each of the above target environment image sets are evaluated and processed to obtain the corresponding label evaluation results. The preset evaluation dimensions include at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. Based on the above label evaluation results, the above candidate environment label information is processed to determine the label, and the target sub-environment label information corresponding to each target environment image set is obtained. Based on a preset hierarchical environment label tree, the above-mentioned target sub-item environment label information is associated to obtain the overall target environment label information corresponding to each target environment image set. Based on the above target sub-environment label information and the above target overall environment label information, the pseudo-labels corresponding to each target environment image set are determined, and the target environment image sets with the above pseudo-labels are added to the preset labeled training set to obtain the target environment image training set. Based on the above target environment image training set, the preset multimodal model is trained under supervision to obtain the target environment recognition model.

[0004] In one possible implementation, the aforementioned preset evaluation dimensions also include at least one of the following: label combination consistency, image quality, and historical data stability; When the above-mentioned preset evaluation dimensions include the above-mentioned model consistency, the above-mentioned label evaluation results include the model consistency scores corresponding to the above-mentioned candidate environment label information. When the aforementioned preset evaluation dimensions include the aforementioned cross-graph evidence consistency, the aforementioned label evaluation results include the cross-graph evidence consistency scores corresponding to the aforementioned candidate environment label information. When the above-mentioned preset evaluation dimensions include the above-mentioned label combination consistency, the above-mentioned label evaluation results include the label combination consistency scores corresponding to the above-mentioned candidate environment label information; When the above-mentioned preset evaluation dimensions include the above-mentioned image quality, the above-mentioned label evaluation results include the image quality scores corresponding to each of the above-mentioned candidate environment label information; Given that the aforementioned preset evaluation dimensions include the aforementioned historical data stability, the aforementioned label evaluation results include the historical data stability scores corresponding to the aforementioned candidate environment label information.

[0005] In one possible implementation, the label determination process is performed on the multiple candidate environment label information based on the label evaluation results to obtain the target sub-environment label information corresponding to each target environment image set, including: Based on the preset label screening conditions and the above label evaluation results, the above multiple candidate environmental label information is screened. The candidate environmental label information that meets the above preset label screening conditions is determined as the first type of candidate environmental label information, and the candidate environmental label information that does not meet the above preset label screening conditions is determined as the second type of candidate environmental label information. Based on the first type of candidate environment label information mentioned above, the target sub-item environment label information of the corresponding target environment image set is determined; If the second type of candidate environment label information exists among the multiple candidate environment label information corresponding to the above target environment image set, the label difference information corresponding to the above target environment image set is determined based on the multiple candidate environment label information corresponding to the above target environment image set. If the above label difference information meets the preset review conditions, the above target environment image set is determined as the image set to be reviewed, and the review sub-item environment label information corresponding to the above image set to be reviewed is obtained. Based on the above-mentioned environmental label information for the review items, the target environmental label information corresponding to the above-mentioned image set to be reviewed is determined.

[0006] In one possible implementation, determining the label difference information corresponding to the target environment image set based on multiple candidate environment label information corresponding to the target environment image set includes: The candidate environment label information corresponding to the above target environment image set is standardized to obtain multiple standard candidate environment label information; Determine the number of recognition models corresponding to each of the above multiple standard candidate environment label information, and determine the target standard candidate environment label information with the largest number of corresponding recognition models; Based on the number of recognition models corresponding to the above target standard candidate environment label information and the total number of the above multiple recognition models, the model divergence degree corresponding to the above target environment image set is determined, and the above model divergence degree is determined as the above label difference information; Under the condition that the aforementioned label difference information meets the preset verification conditions, the aforementioned target environment image set is determined as the image set to be verified, including: If the discrepancy of the above model is greater than or equal to the preset discrepancy threshold, the above target environment image set is determined as the above image set to be reviewed.

[0007] In one possible implementation, the above-mentioned target item environment label information is correlated based on a preset hierarchical environment label tree to obtain the overall target environment label information corresponding to each target environment image set, including: Based on the preset hierarchical environment label tree, determine the label hierarchy information corresponding to the above target sub-item environment label information; Based on the above label hierarchy information and the relationship between different environment labels in the above preset hierarchical environment label tree, the above target item environment label information is combined to obtain multiple candidate overall environment label information corresponding to each target environment image set. Based on the priority relationship between different environment labels in the above-mentioned preset hierarchical environment label tree, the above-mentioned candidate overall environment label information is filtered to obtain the target overall environment label information corresponding to each target environment image set.

[0008] In one possible implementation, the aforementioned target environment image set includes multiple target object environment images arranged in a preset order; The above-mentioned acquisition of multiple target environment image sets corresponding to multiple target objects includes: Obtain the original environmental image set corresponding to each target object among multiple target objects; Image preprocessing is performed on multiple original object environment images included in each of the above original environment image sets to obtain multiple target object environment images; The environmental images of the above multiple target objects are sorted according to a preset order to obtain the target environment image set corresponding to each target object.

[0009] In one possible implementation, the above-mentioned target environment image training set is used to perform supervised fine-tuning training on a preset multimodal model to obtain a target environment recognition model, including: A preset prompt word template is constructed, which is used to instruct the preset multimodal model to predict labels based on the target environment image set; The target environment image sets in the above target environment image training set and the above preset prompt word template are input into the above preset multimodal model to obtain the corresponding recognition output results; Based on the above recognition output results and the training label information corresponding to each target environment image set, the supervised fine-tuning training loss is determined. Based on the above-mentioned supervised fine-tuning training loss, the parameters of the above-mentioned preset multimodal model are updated to obtain the above-mentioned target environment recognition model.

[0010] Secondly, embodiments of this application provide a target environment recognition model inference method, including: Obtain the target merchant's environmental image set; The target merchant environment image set is input into the target environment recognition model to obtain the target object environment recognition result output by the target environment recognition model. The target object environment recognition result is used to characterize the environment category and corresponding sub-environment attributes of the store object corresponding to the target merchant. The target environment recognition model is a recognition model generated by training the target environment recognition model using the target environment recognition model training method as described in the first aspect or any possible implementation of the first aspect.

[0011] Thirdly, embodiments of this application provide a target environment recognition model training apparatus, including: The first acquisition module is used to acquire multiple target environment image sets corresponding to multiple target objects. The target environment image set corresponding to each of the multiple target objects includes multiple target object environment images acquired from multiple perspectives. The recognition module is used to perform feature recognition on each of the above target environment image sets through multiple preset recognition models to obtain multiple candidate environment label information corresponding to each of the above target environment image sets, wherein the multiple recognition models have different model configurations; The evaluation module is used to evaluate multiple candidate environment label information corresponding to each of the above target environment image sets based on a preset evaluation dimension, and obtain the corresponding label evaluation results. The preset evaluation dimension includes at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. The first determining module is used to perform label determining processing on the above multiple candidate environment label information based on the above label evaluation results, so as to obtain the target sub-item environment label information corresponding to each target environment image set. The association module is used to perform association processing on the above target sub-item environment label information based on a preset hierarchical environment label tree, so as to obtain the overall target environment label information corresponding to each target environment image set. The second determining module is used to determine the pseudo-labels corresponding to each of the target environment image sets based on the target sub-environment label information and the target overall environment label information, and to add the target environment image sets with the pseudo-labels to a preset labeled training set to obtain the target environment image training set. The training module is used to perform supervised fine-tuning training of the preset multimodal model based on the above target environment image training set to obtain the target environment recognition model.

[0012] Fourthly, embodiments of this application provide a target environment recognition model inference apparatus, comprising: The second acquisition module is used to acquire the target merchant's environmental image set corresponding to the target merchant; The inference module is used to input the target merchant environment image set into the target environment recognition model to obtain the target object environment recognition result output by the target environment recognition model. The target object environment recognition result is used to characterize the environment category and corresponding sub-environment attributes of the store object corresponding to the target merchant. The aforementioned target environment recognition model is a recognition model generated by training a target environment recognition model using a training method as described in the first aspect or any possible implementation provided by the first aspect.

[0013] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory; The aforementioned memory stores a computer program adapted to be loaded by the aforementioned processor and execute the steps of the method provided by the first aspect of the embodiments of this application or any possible implementation thereof, or the aforementioned computer program adapted to be loaded by the aforementioned processor and execute the steps of the method provided by the second aspect of the embodiments of this application.

[0014] Sixthly, embodiments of this application provide a computer storage medium storing a plurality of instructions, which are adapted to be loaded by a processor and executed by the processor to perform the steps of the method provided by the first aspect of the embodiments of this application or any possible implementation thereof, or the instructions are adapted to be loaded by a processor and executed by the processor to perform the steps of the method provided by the second aspect of the embodiments of this application.

[0015] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the method provided in the first or second aspect of embodiments of this application.

[0016] In this embodiment, multiple target environment image sets corresponding to multiple target objects are acquired. Each target object's target environment image set includes multiple target object environment images acquired from multiple perspectives. This allows the target environment image sets to represent the environmental features of the target objects from different perspectives, thereby reducing the insufficient representation of environmental features caused by limited perspective, partial occlusion, or incomplete environmental information in a single image. Subsequently, multiple preset recognition models are used to perform feature recognition on each of the target environment image sets, obtaining multiple candidate environment label information corresponding to each target environment image set. These multiple recognition models have different model configurations. Furthermore, based on preset evaluation dimensions, the multiple candidate environment label information corresponding to each target environment image set is evaluated to obtain corresponding label evaluation results. These preset evaluation dimensions include at least model consistency and cross-image evidence consistency. Cross-image evidence consistency characterizes the consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. This allows the determination of environment labels to simultaneously consider the recognition consistency between different recognition models and the evidence consistency between multi-view images of the same target object, thereby reducing accidental recognition caused by a single recognition model or a single environment image. The results influence the label determination process, improving the accuracy and reliability of target sub-environment label information. Furthermore, based on the label evaluation results, label determination processing is performed on the multiple candidate environment label information to obtain target sub-environment label information corresponding to each target environment image set. On this basis, based on a preset hierarchical environment label tree, the target sub-environment label information is correlated to obtain the overall target environment label information corresponding to each target environment image set. This overall target environment label information comprehensively represents the overall environmental characteristics of the target object, thereby improving the completeness and consistency of the environment label results. Further, based on the target sub-environment label information and the overall target environment label information, pseudo-labels are determined for each target environment image set, and the target environment image sets with these pseudo-labels are added to a preset labeled training set to obtain a target environment image training set. Based on this target environment image training set, a preset multimodal model is subjected to supervised fine-tuning training to obtain a target environment recognition model. This enables the preset multimodal model to learn the mapping relationship between multi-view environment images and multi-level environment labels, reducing reliance on large-scale manually labeled data while improving the accuracy, reliability, and comprehensiveness of the target environment recognition model in identifying the environmental characteristics of the target object. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An exemplary system architecture diagram of a target environment recognition model training method provided in this application embodiment; Figure 2 A flowchart illustrating another target environment recognition model training method provided in this application embodiment; Figure 3 A flowchart illustrating another target environment recognition model training method provided in an embodiment of this application; Figure 4 A flowchart illustrating another target environment recognition model training method provided in this application embodiment; Figure 5 A flowchart illustrating another target environment recognition model training method provided in this application embodiment; Figure 6 This is a schematic diagram of the overall process of a target environment recognition model training method provided in an embodiment of this application; Figure 7 A flowchart illustrating another target environment recognition model reasoning method provided in this application embodiment; Figure 8 This is a schematic diagram of the structure of a target environment recognition model training device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a target environment recognition model inference device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0020] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus. The term "at least one" in this application means one or more, and the term "multiple" means two or more; for example, multiple second devices means two or more second devices. The terms "system" and "network" are often used interchangeably herein.

[0021] With the continuous accumulation of shop image data on internet platforms such as food delivery and e-commerce, the demand for automatic identification of the shop's environment and attributes based on environmental images from different perspectives, such as the shop's entrance, lobby, and dining area, is constantly increasing. In practical applications, different shops have significant differences in environmental characteristics such as geographical location and dining conditions. If the shop environment identification is inaccurate, it can easily lead to discrepancies between the shop environment classification results and the actual situation, thus affecting subsequent shop selection, classification, and display. Existing environmental image processing methods mostly rely on manual screening or rule verification, which is not only costly and inefficient, but also difficult to stably and comprehensively process large-scale existing data and continuously added image data, resulting in low comprehensiveness and reliability of image recognition.

[0022] For example, in the food delivery scenario, food delivery service platforms typically provide each merchant with an album containing multiple images of the store, including storefront images, lobby or dining area images, which can intuitively reflect the store's geographical location characteristics and overall environment. However, in practical applications, due to the diverse sources of images, varying quality, and high labeling costs, existing methods still have shortcomings in terms of reliability, comprehensiveness, and accuracy in store environment recognition. Besides food delivery scenarios, the embodiments of this application can also be applied to other multi-image environment recognition scenarios, such as indoor space environment recognition, public place environment classification, and product display environment analysis.

[0023] The target environment recognition model training method provided in this application embodiment can be applied to, for example... Figure 1The application environment shown includes a user terminal 10, a server 20, a communication network 30, and a data storage system 40. The user terminal 10, which has an online application (App) installed, communicates with the server 20 corresponding to the online App via the communication network 30. The data storage system 40 stores data that the server 20 needs to process, such as related service data. The data storage system 40 can be integrated onto the server 20 or located on a cloud or other network server.

[0024] In some possible embodiments, server 20 can first acquire multiple target environment image sets corresponding to multiple target objects, wherein each target object's target environment image set includes multiple target object environment images acquired from multiple perspectives; then, it performs feature recognition on each target environment image set using multiple preset recognition models to obtain multiple candidate environment label information corresponding to each target environment image set, wherein the multiple recognition models have different model configurations; next, it evaluates the multiple candidate environment label information corresponding to each target environment image set based on preset evaluation dimensions to obtain corresponding label evaluation results, wherein the preset evaluation dimensions include at least model consistency and cross-image evidence consistency, wherein cross-image evidence consistency is used to characterize the recognition of multiple target object environment images corresponding to the same target object for the same label. The consistency between the results is evaluated; and based on the above label evaluation results, the label determination processing is performed on the above multiple candidate environment label information to obtain the target sub-environment label information corresponding to each target environment image set; based on the preset hierarchical environment label tree, the above target sub-environment label information is associated to obtain the target overall environment label information corresponding to each target environment image set; on this basis, according to the above target sub-environment label information and the above target overall environment label information, the pseudo-labels corresponding to each target environment image set are determined, and the target environment image sets with the above pseudo-labels are added to the preset labeled training set to obtain the target environment image training set; based on the above target environment image training set, the preset multimodal model is trained by supervised fine-tuning (SFT) to obtain the target environment recognition model. Furthermore, the server 20 can obtain the target merchant environment image set corresponding to the target merchant from the user terminal 10; the server 20 inputs the target merchant environment image set into the target environment recognition model to obtain the target object environment recognition result output by the target environment recognition model. The target object environment recognition result is used to characterize the environment category and corresponding sub-environment attributes of the store object corresponding to the target merchant. Then, the target object environment recognition result can be returned to the user terminal 10 so that the user terminal 10 can display the target object environment recognition result to the relevant operation users.

[0025] Understandably, the aforementioned online app can be, but is not limited to, operational applications used for information display, data management, or auxiliary analysis. This app can automatically identify and process target object image information by calling the target environment recognition model deployed on server 20, and obtain the corresponding target object environment recognition results, thereby providing users with visual displays or decision-making references. User terminal 10 can be, but is not limited to, various smartphones, tablets, personal computers, laptops, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. User terminal 10 can be implemented by a single terminal device or a terminal cluster composed of multiple terminals. Server 20 can be implemented by a standalone server or a server cluster composed of multiple servers.

[0026] In one embodiment, such as Figure 2 As shown, a target environment recognition model training method is provided, which can be applied to the aforementioned server, and includes the following steps: S201: Obtain multiple target environment image sets corresponding to multiple target objects, wherein each target object in the multiple target objects' target environment image set includes multiple target object environment images obtained from multiple perspectives.

[0027] The target object can refer to the object entity to be identified or analyzed, specifically it can be a merchant, shop, house, place, commodity or other object with environmental or attribute characteristics.

[0028] Optionally, the target object and the target environment image set can correspond one-to-one. The target environment image set can be image data or a collection of image data used to characterize the features of the corresponding target object. The target environment image set can include multiple target object environment images arranged in a preset order to reflect the external and / or internal environmental features of the target object from multiple perspectives.

[0029] In specific application scenarios, taking food delivery as an example, the target object can be a merchant or store on the food delivery service platform. The aforementioned target environment image set can be the store's photo album image information, such as storefront images, lobby images, or dine-in area images. Multiple target object images can be taken from different perspectives of the store, which can comprehensively reflect the store's geographical location characteristics and overall environment.

[0030] In other application scenarios, the target object can also be a house or interior space. Accordingly, the target environment image set can include images of different areas such as the living room, bedroom, and kitchen, used to represent the spatial layout, decoration style, and functional zoning. In public place environment recognition scenarios, the target object can be a shopping mall, station, hospital, or office area. The target environment image set can include exterior and interior environment images of the location, used to reflect its spatial type and environmental characteristics. In product display or exhibition recognition scenarios, the target object can be a product display area or exhibition space. The target environment image set can include shelf images, exhibition area images, etc., used to identify the display method and exhibition environment characteristics. In industrial inspection scenarios, the target object can be the product to be inspected or the production area. The target environment image set can include multi-angle images of the product or production environment images, used to identify the product status or environmental conditions.

[0031] In some embodiments, multiple sets of original environmental images corresponding to target objects can be obtained from a preset image database, data uploaded by user terminals, data synchronized by third-party image acquisition devices or service systems. Further, the original environmental image sets corresponding to each target object can be filtered and preprocessed to obtain the corresponding target environmental image sets. Thus, images of low quality or irrelevant to the target objects can be removed, and the remaining images can be organized into a target environmental image set according to a preset order.

[0032] For example, in a food delivery scenario, original images such as storefront images, lobby images, and dining area images can be obtained from the store's photo album based on the merchant's identifier. These images are then subjected to quality filtering and normalization processing, and arranged in the order of storefront images first, followed by lobby or dining area images, to obtain the target environment image set corresponding to the merchant.

[0033] S202: By using multiple preset recognition models to perform feature recognition on each of the above target environment image sets, multiple candidate environment label information corresponding to each of the above target environment image sets are obtained, wherein the multiple recognition models have different model configurations.

[0034] The recognition model can be used to extract features and predict labels for each target environment image in the target environment image set. The recognition model can be an image recognition model, a multimodal model or other machine learning model. In this application, multiple recognition models with different structures or different training strategies are preferred.

[0035] Optionally, the candidate environment label information can be a set of candidate sub-item recognition results output by a single recognition model for the target environment image set. That is, for the same target environment image set, different recognition models can output corresponding candidate environment label information, and each candidate environment label information can include one or more sub-item environment labels. For example, recognition model A outputs "shopping mall, has dine-in service, well-decorated", and recognition model B outputs "shopping mall, has dine-in service, well-decorated". Both of these results can be used as candidate environment label information for the target environment image set.

[0036] It should be noted that the sub-environmental labels can be used to characterize the environmental attributes of the target object from different environmental dimensions. These different environmental dimensions can be pre-set according to the specific environmental recognition task, and may include at least one of the following: geographical location dimension, dining conditions dimension, decoration environment dimension, spatial layout dimension, and other dimensions related to the target object's environment. Correspondingly, each environmental dimension can have one or more pre-set label values, and each recognition model can determine the corresponding sub-environmental labels from each environmental dimension based on the environmental features represented by multiple target object environmental images in the target environment image set.

[0037] Optionally, for each recognition model, model training samples including environmental image samples and corresponding environmental label information can be obtained. The environmental image samples undergo preprocessing such as image cropping, resizing, and normalization. The preprocessed environmental image samples are then input into the corresponding recognition model to obtain the model-predicted environmental label information. Based on the difference between the model-predicted environmental label information and the corresponding real environmental label information, the model training loss is determined. The model parameters of the recognition model are iteratively updated based on the model training loss until a preset training termination condition is met, resulting in the trained recognition model. Optionally, to ensure that multiple recognition models have certain differences in environmental feature extraction and environmental label prediction, multiple recognition models can be trained using different model structures, different training data, and / or different training strategies. For example, different recognition models can be trained based on different model network structures, or on different training sample sets, or using different model training parameters and training strategies. This allows multiple recognition models to have different model configurations and to perform environmental feature recognition on the target environmental image set from different feature representation perspectives.

[0038] It is understandable that, due to differences in model configuration among different recognition models, each recognition model may obtain the same sub-environment labels in some environmental dimensions for the same target environment image set, or it may obtain different sub-environment labels.

[0039] In one specific embodiment, for a target environment image set corresponding to each target object, the target environment image set can be input into multiple preset recognition models. Each recognition model extracts features from the multiple target object environment images in the target environment image set and identifies environmental attributes according to multiple preset environmental dimensions to obtain sub-item environmental labels corresponding to each environmental dimension. Optionally, each recognition model can also determine the recognition result of each target object environment image in the target environment image set for the corresponding sub-item environmental label and retain the recognition result corresponding to each target object environment image for subsequent cross-image evidence consistency assessment. Further, one or more sub-item environmental labels obtained by the same recognition model for the target environment image set can be combined to form candidate environment label information corresponding to the recognition model; then, the candidate environment label information output by multiple recognition models can be summarized to obtain multiple candidate environment label information corresponding to the target environment image set.

[0040] For example, in a food delivery scenario, each recognition model extracts features from multiple target object environment images in the target environment image set. This can include identifying geographical location features, dine-in conditions features, and interior decoration features from store image information, thereby obtaining target sub-environment label information such as geographical location labels, dine-in conditions labels, and interior decoration labels. In other scenarios, the above feature extraction can also correspond to space type recognition, functional area recognition, display method recognition, or product status recognition, etc.

[0041] S203: Based on the preset evaluation dimensions, evaluate the multiple candidate environment label information corresponding to each of the above target environment image sets to obtain the corresponding label evaluation results. The preset evaluation dimensions include at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label.

[0042] The preset evaluation dimensions can be multiple evaluation angles or indicators used to assess the quality of candidate environment label information. Optionally, the preset evaluation dimensions may also include at least one of the following: label combination consistency, image quality, and historical data stability. Label combination consistency can be used to characterize whether candidate environment label information conforms to preset hierarchical constraints; image quality can be used to characterize the clarity, completeness, or effectiveness of the target environment image set; historical data stability can be used to characterize the model's misjudgment in historical samples, thereby correcting or weighting the candidate environment label information.

[0043] It is understood that the above label evaluation results can be the evaluation results obtained after comprehensively evaluating each sub-item of the candidate environment label information. The above label evaluation results can be expressed in the form of score value, confidence value, or ranking result. For example, when the above preset evaluation dimension includes the above model consistency, the above label evaluation results include the model consistency score corresponding to each candidate environment label information; when the above preset evaluation dimension includes the above cross-image evidence consistency, the above label evaluation results include the cross-image evidence consistency score corresponding to each candidate environment label information; when the above preset evaluation dimension includes the above label combination consistency, the above label evaluation results include the label combination consistency score corresponding to each candidate environment label information; when the above preset evaluation dimension includes the above image quality, the above label evaluation results include the image quality score corresponding to each candidate environment label information; when the above preset evaluation dimension includes the above historical data stability, the above label evaluation results include the historical data stability score corresponding to each candidate environment label information.

[0044] S204: Based on the above label evaluation results, label determination processing is performed on the above multiple candidate environment label information to obtain the target sub-item environment label information corresponding to each of the above target environment image sets.

[0045] The target sub-item environment label information can be the label information selected from the candidate environment label information. The target sub-item environment label information can include one or more sub-item environment labels, and each sub-item environment label can correspond to an environment dimension.

[0046] It is understandable that candidate environment label information refers to the candidate recognition results output by each recognition model for the target environment image set, while target sub-environment label information is the environment label further determined based on multiple candidate environment label information, combined with the label evaluation results corresponding to preset evaluation dimensions such as model consistency and cross-image evidence consistency. Therefore, compared to the candidate environment label information directly output by a single recognition model, target sub-environment label information can comprehensively reflect the support of multiple recognition models and target object environment images from multiple perspectives for the corresponding environment labels.

[0047] For example, in a shop environment recognition scenario, multiple recognition models output candidate environment label information for the same shop's target environment image set. Recognition model A outputs "Shopping mall, dine-in available, well-decorated," recognition model B outputs "Shopping mall, dine-in available, average decoration," and recognition model C outputs "Shopping mall, dine-in available, well-decorated." After determining the labels by combining the label evaluation results such as model consistency and cross-image evidence consistency corresponding to each candidate environment label information, it can be determined that the target sub-environment labels corresponding to the geographical location dimension include "Shopping mall," the target sub-environment labels corresponding to the dine-in condition dimension include "dine-in available," and the target sub-environment labels corresponding to the decoration environment dimension include "well-decorated." These multiple sub-environment labels can then be collectively used as the target sub-environment label information corresponding to the target environment image set.

[0048] S205: Based on the preset hierarchical environment label tree, perform association processing on the above target sub-item environment label information to obtain the overall target environment label information corresponding to each of the above target environment image sets.

[0049] The preset hierarchical environment tag tree can be a tree-shaped tag structure pre-constructed based on the hierarchical, subordinate, and / or related relationships between different environment tags. It can be used to organize sub-environment tags under different environment dimensions and establish the correspondence between target sub-environment tags and the overall environment tags.

[0050] Optionally, the pre-defined hierarchical environment tag tree can include multiple tag levels, with different tag levels corresponding to different granularities of environment attributes. Lower-level tags can be used to describe the environmental attributes of a target object within a specific environmental dimension, while higher-level tags can be used to comprehensively describe the overall environmental category of the target object based on one or more lower-level tags. At least one of the following relationships can be pre-configured between environment tags at different levels: tag hierarchy, tag association, tag mapping, and tag priority.

[0051] For example, in a shop environment recognition scenario, a pre-defined hierarchical environment label tree can include sub-environment dimensions such as geographical location, dine-in conditions, and interior design. The geographical location dimension can include sub-environment labels such as "shopping mall" and "street-side," the dine-in conditions dimension can include sub-environment labels such as "dine-in available" and "no dine-in available," and the interior design dimension can include sub-environment labels such as "well-decorated" and "average-decorated." Furthermore, the association between combinations of these different sub-environment labels and the corresponding overall environment labels can be pre-established. For example, a combination of multiple sub-environment labels such as "shopping mall," "dine-in available," and "well-decorated" can be associated with the corresponding overall environment category.

[0052] Optionally, the overall target environment label information can be used to comprehensively represent the overall environmental category of the target object. Compared with the target component environment label information, which mainly describes the environmental attributes of the target object in a specific environmental dimension, the overall target environment label information is used to comprehensively express the environmental attributes corresponding to multiple environmental dimensions. In other words, the overall target environment label information is not directly output by the recognition model for a single image, but is further determined based on the already determined target component environment label information through the correlation between multiple component environment labels.

[0053] In some specific application scenarios, taking the food delivery scenario as an example, the above-mentioned overall target environment label information can be used to characterize the environmental characteristics of the store object. Among them, the above-mentioned target sub-environment label information can include geographical location related labels, dine-in condition related labels, and decoration environment related labels, etc. The above-mentioned overall target environment label information can be determined based on the above-mentioned multiple target sub-environment label information to characterize the overall category of the store environment.

[0054] In other application scenarios, the aforementioned overall target environment label information can also be represented in a structured form. For example, in a house or interior space recognition scenario, the aforementioned target sub-environment label information may include space type labels, decoration style labels, and functional zoning labels, and the aforementioned overall target environment label information can be used to characterize the overall type or level of the house; in a public place environment recognition scenario, the aforementioned target sub-environment label information may include place functional attribute labels and spatial structure labels, and the aforementioned overall target environment label information can be used to characterize the place type; in a product display or arrangement recognition scenario, the aforementioned target sub-environment label information may include display method labels and display environment labels, and the aforementioned overall target environment label information can be used to characterize the overall arrangement type.

[0055] S206: Based on the above target sub-environment label information and the above target overall environment label information, determine the pseudo-labels corresponding to each of the above target environment image sets, and add the target environment image sets with the above pseudo-labels to the preset labeled training set to obtain the target environment image training set.

[0056] Optionally, the target component environment label information and the target overall environment label information can be combined according to a preset label structure to obtain pseudo-labels corresponding to the target environment image set. The preset label structure mentioned above may include component environment label fields and overall environment label fields.

[0057] The labeled training set may include a set of labeled target environment images and corresponding label information, wherein the label information may come from manual annotation and / or model prediction results.

[0058] It is understood that the target environment image training set can include both existing manually labeled training samples and training samples with pseudo-labels automatically generated through the processing described above in this application. By using structured pseudo-labels containing target sub-environment label information and target overall environment label information to expand the training data, the number of environment image samples that can be used for subsequent supervised fine-tuning training can be increased, while enabling the training labels to simultaneously represent fine-grained environmental attributes and overall environmental categories.

[0059] S207: Based on the above target environment image training set, supervised fine-tuning training of the preset multimodal model is performed to obtain the target environment recognition model.

[0060] The preset multimodal model can be a model capable of jointly modeling and processing multimodal data. Multimodal data can include image data and text prompts, etc. The preset multimodal model can be a multimodal model that combines a large language model and a visual model.

[0061] It is understandable that the target environment recognition model can be a model obtained by supervising and fine-tuning the above-mentioned preset multimodal model, which is used to recognize and process new target environment image sets and output corresponding target label information.

[0062] In this embodiment, multiple target environment image sets corresponding to multiple target objects are acquired. Each target object's target environment image set includes multiple target object environment images acquired from multiple perspectives. This allows the target environment image sets to represent the environmental features of the target objects from different perspectives, thereby reducing the insufficient representation of environmental features caused by limited perspective, partial occlusion, or incomplete environmental information in a single image. Subsequently, multiple preset recognition models are used to perform feature recognition on each of the target environment image sets, obtaining multiple candidate environment label information corresponding to each target environment image set. These multiple recognition models have different model configurations. Furthermore, based on preset evaluation dimensions, the multiple candidate environment label information corresponding to each target environment image set is evaluated to obtain corresponding label evaluation results. These preset evaluation dimensions include at least model consistency and cross-image evidence consistency. Cross-image evidence consistency characterizes the consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. This allows the determination of environment labels to simultaneously consider the recognition consistency between different recognition models and the evidence consistency between multi-view images of the same target object, thereby reducing accidental recognition caused by a single recognition model or a single environment image. The results influence the label determination process, improving the accuracy and reliability of target sub-environment label information. Furthermore, based on the label evaluation results, label determination processing is performed on the multiple candidate environment label information to obtain target sub-environment label information corresponding to each target environment image set. On this basis, based on a preset hierarchical environment label tree, the target sub-environment label information is correlated to obtain the overall target environment label information corresponding to each target environment image set. This overall target environment label information comprehensively represents the overall environmental characteristics of the target object, thereby improving the completeness and consistency of the environment label results. Further, based on the target sub-environment label information and the overall target environment label information, pseudo-labels are determined for each target environment image set, and the target environment image sets with these pseudo-labels are added to a preset labeled training set to obtain a target environment image training set. Based on this target environment image training set, a preset multimodal model is subjected to supervised fine-tuning training to obtain a target environment recognition model. This enables the preset multimodal model to learn the mapping relationship between multi-view environment images and multi-level environment labels, reducing reliance on large-scale manually labeled data while improving the accuracy, reliability, and comprehensiveness of the target environment recognition model in identifying the environmental characteristics of the target object.

[0063] In one embodiment, in S203, the above-mentioned evaluation processing of multiple candidate environment label information corresponding to each target environment image set based on a preset evaluation dimension to obtain the corresponding label evaluation result may include the following steps: based on the recognition results of the multiple recognition models for the same environment label, performing model consistency evaluation on the candidate environment label information to obtain the model consistency score corresponding to the candidate environment label information; and / or, based on the recognition results of multiple target object environment images in the target environment image set for the same environment label, performing cross-image evidence consistency evaluation on the candidate environment label information to obtain the cross-image evidence consistency score corresponding to the candidate environment label information; and / or, based on the candidate environment label... The combination relationship between different sub-environmental labels in the information and the association relationship between different environmental labels in the preset hierarchical environmental label tree are used to evaluate the label combination consistency of the candidate environmental label information and obtain the label combination consistency score corresponding to the candidate environmental label information; and / or, based on the image quality information corresponding to multiple target object environmental images in the target environmental image set, the image quality of the candidate environmental label information is evaluated and the image quality score corresponding to the candidate environmental label information is obtained; and / or, based on the historical recognition results of the recognition model for historical environmental image samples, the historical data stability of the candidate environmental label information is evaluated and the historical data stability score corresponding to the candidate environmental label information is obtained.

[0064] Specifically, at least one of the above-mentioned model consistency score, the above-mentioned cross-image evidence consistency score, the above-mentioned label combination consistency score, the above-mentioned image quality score, and the above-mentioned historical data stability score can be determined as the label evaluation result corresponding to the above-mentioned candidate environment label information; or, at least two of the evaluation scores can be fused to obtain the comprehensive evaluation result corresponding to the above-mentioned candidate environment label information, and the comprehensive evaluation result can be determined as the above-mentioned label evaluation result.

[0065] Specifically, regarding model consistency, the number of recognition models that output the same candidate environment label information for the same target environment image set can be counted. The model consistency score corresponding to the candidate environment label information is then determined based on the ratio between this number and the total number of recognition models participating in the recognition process. The more recognition models output a particular candidate environment label information, the more consistent the recognition results of multiple recognition models for that target environment image set, and the higher the corresponding model consistency score. For example, in a food delivery scenario, for a target environment image set corresponding to the same shop, if four out of five recognition models output the candidate environment label information "shopping mall, dine-in available, well-decorated," and only one recognition model outputs a different candidate environment label information, then the model consistency score corresponding to that candidate environment label information can be determined as 4 / 5, or 0.8, based on the ratio between the number of recognition models outputting the same candidate environment label information and the total number of recognition models.

[0066] Optionally, for cross-image evidence consistency, the recognition results of multiple target environment images in the same target environment image set for the same sub-environment label can be obtained separately. The corresponding cross-image evidence consistency score is determined based on the relationship between the number of target environment images supporting the sub-environment label and the total number of target environment images in the target environment image set. The more target environment images supporting the same sub-environment label, the more consistent the recognition evidence provided by target environment images from different perspectives for that environmental attribute, and the higher the corresponding cross-image evidence consistency score. For example, in a food delivery scenario, the target environment image set includes storefront images, lobby images, and dine-in area images. If multiple target environment images can all provide environmental features supporting the sub-environment label "dine-in available," then the cross-image evidence consistency score corresponding to "dine-in available" is high; if different target environment images obtain conflicting recognition results for "dine-in available," then the corresponding cross-image evidence consistency score is low.

[0067] Optionally, regarding tag combination consistency, the tag combination consistency of the candidate environment tag information can be evaluated based on the combination relationships between different sub-environment tags in the candidate environment tag information and the association relationships between different environment tags in the preset hierarchical environment tag tree, thus obtaining the corresponding tag combination consistency score. Specifically, it can be determined whether the combination of each sub-environment tag in the candidate environment tag information conforms to the pre-set subordinate relationships, mapping relationships, combination association relationships, and / or mutual exclusion relationships in the preset hierarchical environment tag tree. When there are mutual exclusion relationships or the pre-set association relationships between multiple sub-environment tags in the candidate environment tag information, the corresponding tag combination consistency score can be determined to be low; when the combination of multiple sub-environment tags conforms to the association relationships between corresponding environment tags in the preset hierarchical environment tag tree, the corresponding tag combination consistency score can be determined to be high. For example, if the candidate environment label information includes multiple sub-environment labels corresponding to different environment dimensions, and two of the sub-environment labels are set to a mutually exclusive relationship in the preset hierarchical environment label tree, it can be determined that there is a conflict in the label combination in the candidate environment label information, and thus the label combination consistency score corresponding to the candidate environment label information is low; conversely, if each sub-environment label in the candidate environment label information conforms to the corresponding association relationship in the preset hierarchical environment label tree, it can be determined that the corresponding label combination consistency score is high.

[0068] Optionally, regarding image quality, image quality can be evaluated based on at least one image quality feature from multiple target environment images in the target environment image set, such as image clarity, completeness, brightness, occlusion, and effective area ratio, to obtain a corresponding image quality score. For example, in a food delivery scenario, if the storefront image, lobby image, or dining area image used for environmental feature recognition is severely blurred, occluded, or has a small effective environmental area, the reliability of the candidate environment label information obtained based on the corresponding target environment image is relatively low, and the corresponding image quality score can be determined to be low. Conversely, if the target environment image is clear, complete, and contains sufficient environmental features, the corresponding image quality score can be determined to be high.

[0069] Optionally, regarding historical data stability, the recognition model can obtain historical recognition results for the same or similar environmental labels in historical environmental image samples. These historical recognition results may include at least one of historical prediction accuracy, historical misjudgment rate, and historical prediction result fluctuation. Based on these historical recognition results, the candidate environmental label information is evaluated for historical data stability to obtain a corresponding historical data stability score. Specifically, the higher the historical prediction accuracy, the lower the historical misjudgment rate, and / or the smaller the historical prediction result fluctuation for the corresponding environmental label, the higher the corresponding historical data stability score. For example, in a food delivery scenario, if historical environmental image samples show that the recognition model's recognition results for the "dine-in available" sub-item environmental label are relatively stable and have a high prediction accuracy over a long period, then the historical data stability score for this sub-item environmental label can be determined to be high. Conversely, if the recognition model frequently confuses "well-decorated" and "average-decorated" in historical environmental image samples, then the historical data stability score for the corresponding candidate environmental label information can be determined to be low.

[0070] In this embodiment, candidate environment label information is comprehensively evaluated from multiple assessment dimensions, including model consistency, cross-image evidence consistency, label combination consistency, image quality, and historical data stability. This approach simultaneously considers the consistency between different recognition models, the consistency of evidence between multi-view images of the same target object, and the rationality of the candidate environment label combination, image quality, and historical prediction stability. This avoids determining environment labels based solely on a single model, a single image, or a single evaluation metric. Furthermore, by fusing the evaluation scores corresponding to each assessment dimension, the reliability of candidate environment label information can be more comprehensively characterized. This reduces the impact of accidental model misjudgments, insufficient local image information, and unreasonable label combinations on the label determination results, thereby improving the accuracy, stability, and reliability of subsequent target sub-item environment label information and pseudo-labels.

[0071] In one embodiment, such as Figure 3 As shown, another method for training a target environment recognition model is provided, including the following steps: S301: Obtain multiple target environment image sets corresponding to multiple target objects, wherein each target object in the multiple target objects' target environment image set includes multiple target object environment images obtained from multiple perspectives.

[0072] Specifically, S301 is the same as S201, and will not be repeated here.

[0073] S302: By using multiple preset recognition models to perform feature recognition on each of the above target environment image sets, multiple candidate environment label information corresponding to each of the above target environment image sets are obtained, wherein the multiple recognition models have different model configurations.

[0074] Specifically, S302 is the same as S202, and will not be repeated here.

[0075] S303: Based on the preset evaluation dimensions, evaluate the multiple candidate environment label information corresponding to each of the above target environment image sets to obtain the corresponding label evaluation results. The preset evaluation dimensions include at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label.

[0076] Specifically, S303 is the same as S203, and will not be repeated here.

[0077] S304: Based on the preset label screening conditions and the above label evaluation results, the above multiple candidate environmental label information is screened, and the candidate environmental label information that meets the above preset label screening conditions is determined as the first type of candidate environmental label information, and the candidate environmental label information that does not meet the above preset label screening conditions is determined as the second type of candidate environmental label information.

[0078] The preset label selection criteria can be used to determine whether candidate environment label information has sufficient reliability based on the label evaluation results. These preset label selection criteria can be set according to the actual environment recognition task and different evaluation dimensions, and may include at least one of the following: score threshold conditions, confidence threshold conditions, and ranking conditions. When the label evaluation results include evaluation scores corresponding to multiple evaluation dimensions, the multiple evaluation scores can be fused first to obtain a comprehensive evaluation result corresponding to the candidate environment label information, and then the preset label selection criteria can be determined based on this comprehensive evaluation result.

[0079] It should be noted that the first type of candidate environment label information refers to candidate environment label information whose label evaluation results meet the preset label screening conditions. That is, after evaluations such as model consistency and cross-image evidence consistency, it is determined to be candidate environment label information with high reliability. The first type of candidate environment label information can be directly used as the data basis for determining the target sub-item environment label information. The second type of candidate environment label information refers to candidate environment label information whose label evaluation results do not meet the preset label screening conditions. The second type of candidate environment label information may indicate that the reliability of the corresponding candidate environment label is insufficient, or that there are still certain differences in the recognition results of multiple recognition models and multiple target object environment images for the corresponding environmental attributes. Therefore, further label difference analysis can be performed on the corresponding target environment image set to determine whether a review process is needed.

[0080] For example, when the label evaluation results are represented by a comprehensive score, the comprehensive score corresponding to the candidate environmental label information can be compared with a preset scoring threshold. When the comprehensive score is greater than or equal to the preset scoring threshold, the corresponding candidate environmental label information is determined as the first type of candidate environmental label information; when the comprehensive score is less than the preset scoring threshold, the corresponding candidate environmental label information is determined as the second type of candidate environmental label information. Alternatively, the candidate environmental label information can be sorted from highest to lowest according to the label evaluation results corresponding to each candidate environmental label information, and the candidate environmental label information within the preset range can be determined as the first type of candidate environmental label information, and the remaining candidate environmental label information can be determined as the second type of candidate environmental label information.

[0081] For example, in a shop environment recognition scenario, for a target shop's corresponding image set, multiple recognition models output multiple candidate environment label information. If the model consistency score and cross-image evidence consistency score corresponding to "shopping mall, has dine-in service, and is well-decorated" are both high, and the comprehensive evaluation result reaches a preset reliability threshold, then this candidate environment label information can be identified as the first type of candidate environment label information; if the comprehensive evaluation result corresponding to another candidate environment label information is low, then it can be identified as the second type of candidate environment label information.

[0082] S305: Based on the first type of candidate environment label information mentioned above, determine the target sub-item environment label information of the corresponding target environment image set.

[0083] Optionally, the sub-item environment labels corresponding to each preset environment dimension can be obtained from the first type of candidate environment label information, and the sub-item environment labels that meet the reliability requirements can be determined as the target sub-item environment labels corresponding to the corresponding environment dimension, thereby obtaining the target sub-item environment label information corresponding to the target environment image set.

[0084] For example, if the first type of candidate environmental label information includes "shopping mall" corresponding to the geographical location dimension, "dine-in available" corresponding to the dine-in conditions dimension, and "well-decorated" corresponding to the decoration environment dimension, then "shopping mall", "dine-in available" and "well-decorated" can be determined as the target sub-item environmental labels corresponding to the corresponding environmental dimensions, and the above multiple target sub-item environmental labels can be jointly determined as the target sub-item environmental label information corresponding to the target environment image set.

[0085] S306: If the second type of candidate environment label information exists among the multiple candidate environment label information corresponding to the target environment image set, determine the label difference information corresponding to the target environment image set based on the multiple candidate environment label information corresponding to the target environment image set.

[0086] Label difference information can be used to characterize the degree of difference between multiple candidate environment label information output by multiple recognition models for the same target environment image set. Label difference information can be represented by numerical values, ranks, or other data forms that can characterize the degree of difference between the outputs of multiple recognition models.

[0087] In one embodiment, in S306, determining the label difference information corresponding to the target environment image set based on multiple candidate environment label information corresponding to the target environment image set may include the following steps: standardizing the multiple candidate environment label information corresponding to the target environment image set to obtain multiple standard candidate environment label information; determining the number of recognition models corresponding to each standard candidate environment label information among the multiple standard candidate environment label information, and determining the target standard candidate environment label information with the largest number of corresponding recognition models; determining the model divergence degree corresponding to the target environment image set based on the number of recognition models corresponding to the target standard candidate environment label information and the total number of the multiple recognition models, and determining the model divergence degree as the label difference information.

[0088] Understandably, since different recognition models may use different textual expressions to describe semantically similar or identical environmental labels, multiple candidate environmental label information can be uniformly processed before performing label difference analysis. This standardization process can include at least one of label text normalization, synonym mapping, label format conversion, and standard label mapping. For example, for labels with similar semantics such as "well-decorated," "relatively well-decorated," and "relatively well-decorated environment," they can be uniformly mapped to the same standard candidate environmental label information according to a pre-configured standard label mapping relationship, thereby avoiding erroneously increasing the degree of difference between multiple recognition models due to different label textual expressions.

[0089] Optionally, model divergence can be used to quantify the degree of difference between the environmental label recognition results of multiple recognition models for the same target environment image set. Model divergence can be determined based on the distribution of multiple standard candidate environmental label information. For example, the number of recognition models corresponding to each standard candidate environmental label information can be counted, further identifying the target standard candidate environmental label information with the largest number of corresponding recognition models, and the model divergence can be determined based on the relationship between the number of recognition models corresponding to the aforementioned target standard candidate environmental label information and the total number of recognition models participating in the recognition. Generally speaking, the more recognition models that support the same standard candidate environmental label information, the more consistent the recognition results of multiple recognition models are, and the lower the corresponding model divergence; conversely, the more dispersed the standard candidate environmental label information output by multiple recognition models, the higher the model divergence.

[0090] For example, the model divergence can be determined as: 1 minus the ratio between the number of models corresponding to the highest percentage of standard candidate environment label information and the total number of recognition models. If the 5 recognition models output 3 different types of standard candidate environment label information respectively, and the highest percentage is only 2 / 5, then the model divergence can be 1-2 / 5=0.6.

[0091] S307: If the above label difference information meets the preset review conditions, the above target environment image set is determined as the image set to be reviewed, and the review sub-item environment label information corresponding to the above image set to be reviewed is obtained.

[0092] The preset verification conditions can be used to determine whether the target environment image set needs further label verification. The preset verification conditions can be set based on label difference information. For example, a preset divergence threshold can be set. When the model divergence is greater than or equal to the preset divergence threshold, it can be determined that there is a large difference between the recognition results of multiple recognition models for the target environment image set, thereby determining that the label difference information meets the preset verification conditions.

[0093] In one embodiment, determining the target environment image set as the image set to be reviewed when the label difference information meets the preset review conditions includes: determining the target environment image set as the image set to be reviewed when the model divergence degree is greater than or equal to the preset divergence degree threshold.

[0094] It is understandable that verifying the sub-environmental label information can be used to characterize the environmental attributes of the target object in the corresponding environmental dimension. The above verification process can be implemented through a pre-set verification system. For example, the image set to be verified and the corresponding multiple candidate environmental label information can be sent to the verification system. The verification system then re-verifies the sub-environmental labels that differ based on the environmental features represented by the multiple target object environmental images in the image set to be verified, and returns the corresponding verified sub-environmental label information. Optionally, the above verification process may include manual verification.

[0095] S308: Based on the above-mentioned review item environmental label information, determine the target item environmental label information corresponding to the above-mentioned image set to be reviewed.

[0096] Optionally, after obtaining the environmental label information for each environmental dimension of the image set to be reviewed, the environmental labels for each environmental dimension in the aforementioned environmental label information can be obtained, and each of these environmental labels can be determined as the target environmental label for the corresponding environmental dimension. Then, target environmental label information for the image set to be reviewed can be formed based on one or more target environmental labels. Therefore, for target environmental image sets with significant differences in model recognition results, the environmental attributes of their components can be further confirmed through review processing, making the target environmental labels used to determine the overall target environmental label information more reliable.

[0097] For example, in a shop environment recognition scenario, if multiple recognition models have relatively consistent recognition results for the geographical location and dine-in conditions of a certain shop, but output different labels such as "well-decorated" and "average-decorated" for the decoration environment, and the target environment image set corresponding to the shop is determined as the image set to be reviewed due to the high degree of model divergence; if after review it is determined that the decoration environment label corresponding to the shop is "well-decorated", then the target sub-item environment label information corresponding to the image set to be reviewed can be determined based on the review sub-item environment labels such as "well-decorated".

[0098] S309: Based on the preset hierarchical environment label tree, the above-mentioned target sub-item environment label information is associated to obtain the overall target environment label information corresponding to each of the above-mentioned target environment image sets.

[0099] Specifically, S309 is the same as S205, and will not be repeated here.

[0100] S310: Based on the above target sub-environment label information and the above target overall environment label information, determine the pseudo-labels corresponding to each of the above target environment image sets, and add the target environment image sets with the above pseudo-labels to the preset labeled training set to obtain the target environment image training set.

[0101] Specifically, S310 is the same as S206, and will not be repeated here.

[0102] S311: Based on the above target environment image training set, supervised fine-tuning training of the preset multimodal model is performed to obtain the target environment recognition model.

[0103] Specifically, S311 is the same as S207, and will not be repeated here.

[0104] This application embodiment filters multiple candidate environment label information based on label evaluation results. Highly reliable candidate environment labels can be directly used to determine the target sub-environment label information. Candidate environment labels with insufficient reliability or significant differences in model recognition results undergo further difference analysis and verification correction, thereby reducing the possibility of low-quality or erroneous candidate labels being directly adopted and improving the accuracy and reliability of the target sub-environment label information. Simultaneously, by standardizing the candidate environment label information output by multiple recognition models and determining the model divergence degree, false divergences caused by different model label representations can be reduced, making the selection of verification objects more accurate. Furthermore, by pre-setting a hierarchical environment label tree to associate the target sub-environment label information with the overall target environment label information, and constructing pseudo-labels based on the target sub-environment label information and the overall target environment label information, the integrity and credibility of the pseudo-labels can be improved. This enhances the label quality of the target environment image training set, providing more reliable training samples for subsequent supervised fine-tuning of the multimodal model, and ultimately improving the accuracy, stability, and reliability of the target environment recognition model in recognizing the target object's sub-environment attributes and overall environment category.

[0105] In one embodiment, such as Figure 4 As shown, another method for training a target environment recognition model is provided, including the following steps: S401: Obtain multiple target environment image sets corresponding to multiple target objects, wherein each target object in the multiple target objects' target environment image set includes multiple target object environment images obtained from multiple perspectives.

[0106] Specifically, S401 is the same as S201, and will not be repeated here.

[0107] S402: By using multiple preset recognition models to perform feature recognition on each of the above target environment image sets, multiple candidate environment label information corresponding to each of the above target environment image sets are obtained, wherein the multiple recognition models have different model configurations.

[0108] Specifically, S402 is the same as S202, and will not be repeated here.

[0109] S403: Based on the preset evaluation dimensions, evaluate the multiple candidate environment label information corresponding to each of the above target environment image sets to obtain the corresponding label evaluation results. The preset evaluation dimensions include at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label.

[0110] Specifically, S403 is the same as S203, and will not be repeated here.

[0111] S404: Based on the above label evaluation results, label determination processing is performed on the above multiple candidate environment label information to obtain the target sub-item environment label information corresponding to each of the above target environment image sets.

[0112] Specifically, S404 is the same as S204, and will not be repeated here.

[0113] S405: Based on the preset hierarchical environment label tree, determine the label level information corresponding to the above target sub-item environment label information.

[0114] The preset hierarchical environment tag tree can be used to organize multiple environment tags according to different levels and record the relationships between different environment tags. This preset hierarchical environment tag tree can include multiple environment tag nodes, each corresponding to a different environment tag, and set at different levels according to the corresponding environment dimension, tag granularity, and / or the relationships between tags. Environment tags at different levels can have subordinate, mapping, or combined relationships, so that specific sub-environment tags can be further associated with environment tags representing the overall environment category according to the preset tag structure.

[0115] Optionally, in the shop environment recognition scenario, the pre-defined hierarchical environment label tree can include label nodes corresponding to different environmental dimensions such as geographical location, dine-in conditions, and decoration environment. For example, environment labels related to geographical location can include "shopping mall" and "streetside," environment labels related to dine-in conditions can include "dine-in available" and "no dine-in available," and environment labels related to decoration environment can include "good decoration" and "average decoration." The above-mentioned different sub-environment labels can be further associated with the corresponding overall environment labels.

[0116] It is understood that tag hierarchy information can be used to characterize the tag hierarchy and / or environment dimension to which the corresponding sub-item environment tag belongs. Specifically, tag hierarchy information may include at least one of the following: the tag's hierarchy, the tag's environment dimension, and the tag's node position in a predefined hierarchical environment tag tree.

[0117] For example, for the target sub-item environmental label information "shopping mall, dine-in available, well-decorated", the preset hierarchical environmental label tree can be used to determine that "shopping mall" corresponds to the geographical location dimension, "dine-in available" corresponds to the dine-in condition dimension, and "well-decorated" corresponds to the decoration environment dimension, thereby obtaining the label level information corresponding to each target sub-item environmental label.

[0118] In one specific embodiment, the target sub-item environment tags included in the target sub-item environment tag information can be queried in a preset hierarchical environment tag tree to determine the environment tag node corresponding to each target sub-item environment tag. Based on the position of each environment tag node in the preset hierarchical environment tag tree, the tag level and environment dimension to which the corresponding target sub-item environment tag belongs can be determined, thereby obtaining the tag level information corresponding to the target sub-item environment tag information. Thus, the multiple target sub-item environment tags identified above can be categorized according to a preset tag structure.

[0119] S406: Based on the above-mentioned label hierarchy information and the relationship between different environment labels in the above-mentioned preset hierarchical environment label tree, the above-mentioned target sub-item environment label information is combined and processed to obtain multiple candidate overall environment label information corresponding to each of the above-mentioned target environment image sets.

[0120] The aforementioned relationships may include at least one of the following: combination relationships between sub-environmental tags under different environmental dimensions, subordinate relationships between environmental tags at different levels, and mapping relationships between sub-environmental tags and the overall environmental tag. Through these relationships, the overall environmental tag corresponding to a combination of one or more target sub-environmental tags can be determined.

[0121] Understandably, candidate overall environment label information can be used as label information that may characterize the overall environment category of the target object. Since the same set of target sub-environment labels may correspond to multiple overall environment labels in different combinations or mapping relationships, multiple candidate overall environment label information can be obtained first, and then the final target overall environment label information can be determined from them through subsequent steps.

[0122] Specifically, based on the tag hierarchy information, target sub-environment tags belonging to different tag levels or different environmental dimensions can be combined to obtain one or more sub-environment tag combinations. For each sub-environment tag combination, the association relationship corresponding to the above sub-environment tag combination is queried in the preset hierarchical environment tag tree, and the overall environment tag that can establish a mapping relationship with the corresponding sub-environment tag combination is determined based on the above association relationship. Then, the determined overall environment tag is used as candidate overall environment tag information.

[0123] S407: Based on the priority relationship between different environment labels in the above-mentioned preset hierarchical environment label tree, the above-mentioned multiple candidate overall environment label information is filtered to obtain the target overall environment label information corresponding to each target environment image set.

[0124] The priority relationship can be used to characterize the relative importance of different environmental labels in determining the overall environmental label. This priority relationship can be set according to the distinguishing ability of different environmental dimensions for the overall environmental category, scene recognition requirements, and / or label importance. When multiple sub-environmental labels can be associated with different candidate overall environmental labels, the priority relationship can be used to determine the screening order of different candidate overall environmental labels, thereby reducing the multiple solutions or conflicts that arise when multiple label combinations can simultaneously form an overall environmental label.

[0125] Specifically, one or more target sub-environmental labels associated with each candidate overall environment label information can be obtained, and the priority of each candidate overall environment label information can be determined according to the priority relationship between different environment labels in a preset hierarchical environment label tree. Multiple candidate overall environment label information are then sorted and / or filtered according to the aforementioned priority, and the candidate overall environment label information whose priority meets the preset requirements is determined as the target overall environment label information corresponding to the target environment image set. Optionally, the candidate overall environment label information with the highest priority can be determined as the target overall environment label information.

[0126] For example, in the food delivery scenario, the priority relationship can be set as: geographical location > dine-in conditions > decoration environment.

[0127] In this case, "shopping mall restaurants" can be selected as the final target overall environment label information because they also meet the high priority of the "shopping mall" label, which is higher than "dine-in restaurants".

[0128] S408: Based on the above target sub-environment label information and the above target overall environment label information, determine the pseudo-labels corresponding to each of the above target environment image sets, and add the target environment image sets with the above pseudo-labels to the preset labeled training set to obtain the target environment image training set.

[0129] Specifically, S408 is the same as S206, and will not be repeated here.

[0130] S409: Based on the above target environment image training set, supervised fine-tuning training of the preset multimodal model is performed to obtain the target environment recognition model.

[0131] Specifically, S409 is the same as S207, and will not be repeated here.

[0132] In this embodiment, the target sub-environmental label information is hierarchically categorized by a preset hierarchical environment label tree. Multiple candidate overall environment label information is generated by combining the relationships between different environment labels. This ensures that the conversion process from sub-environmental labels to overall environment labels has clear hierarchical and relational constraints, reducing the problem of unreasonable label combinations. Furthermore, by filtering the multiple candidate overall environment label information based on the priority relationships between different environment labels, the target overall environment label information that better conforms to the preset label rules can be determined when multiple possible overall labels exist, thereby improving the standardization and consistency of overall environment label generation. Based on this, the target sub-environmental label information and the target overall environment label information are added to the labeled training set as pseudo-labels for the target environment image set. This allows the training samples to simultaneously contain fine-grained environmental attributes and overall environment category information, improving the completeness and effectiveness of the training data labels. This, in turn, enhances the target environment recognition model's ability to learn complex environmental features and the accuracy and reliability of the target object environment recognition results.

[0133] In one embodiment, such as Figure 5 As shown, another method for training a target environment recognition model is provided, including the following steps: S501: Obtain the original environment image set corresponding to each target object among multiple target objects.

[0134] The original environmental image set can be a collection of unprocessed environmental image data acquired for the same target object. Specifically, the original environmental image set can include a collection of image data acquired by a user terminal, stored on a platform, or provided by a third-party system, which may contain multiple original object environment images reflecting different perspectives or different areas of the target object.

[0135] S502: Perform image preprocessing on the multiple original object environment images included in each of the above-mentioned original environment image sets to obtain multiple target object environment images.

[0136] The aforementioned original object environment image can be a single environment image from the set of original environment images, used to characterize a local feature of the target object or a certain environmental perspective feature. For example, in a food delivery scenario, the aforementioned original object environment image can be an image of the storefront, a lobby, or a dining area, etc.

[0137] Optionally, the aforementioned image preprocessing may refer to normalizing the original object environment image. Specifically, the image preprocessing may include at least one of the following: duplicate image filtering, blurred image filtering, invalid image filtering, image cropping, image scaling, image normalization, and image format conversion. For example, blurred or severely occluded images can be removed, and the remaining images can be resized to obtain a target object environment image suitable for model input.

[0138] It is understood that the aforementioned target object environment image can refer to the image data retained after the above image preprocessing, which is used for subsequent model recognition processing. Compared with the original object environment image, the aforementioned target object environment image has higher effectiveness and consistency.

[0139] S503: Sort the above-mentioned multiple target object environment images according to a preset order to obtain the target environment image set corresponding to each of the above-mentioned target objects.

[0140] The aforementioned preset order can be a rule for arranging images of the target object's environment. Specifically, the preset order can be set based on image type, shooting angle, time sequence, or image quality. For example, it can be arranged in the order of "exterior images first, interior environment images second".

[0141] For example, in a food delivery scenario, the storefront image can be arranged first, followed by the lobby or dining area image, thus forming a target environment image set with a structured order.

[0142] S504: By using multiple preset recognition models to perform feature recognition on each of the above target environment image sets, multiple candidate environment label information corresponding to each of the above target environment image sets are obtained, wherein the multiple recognition models have different model configurations.

[0143] Specifically, S504 is the same as S202, and will not be repeated here.

[0144] S505: Based on the preset evaluation dimensions, evaluate and process the multiple candidate environment label information corresponding to each of the above target environment image sets to obtain the corresponding label evaluation results. The preset evaluation dimensions include at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label.

[0145] Specifically, S505 is the same as S203, and will not be repeated here.

[0146] S506: Based on the above label evaluation results, label determination processing is performed on the above multiple candidate environment label information to obtain the target sub-item environment label information corresponding to each of the above target environment image sets.

[0147] Specifically, S506 is the same as S204, and will not be repeated here.

[0148] S507: Based on the preset hierarchical environment label tree, the above-mentioned target sub-item environment label information is associated to obtain the overall target environment label information corresponding to each of the above-mentioned target environment image sets.

[0149] Specifically, S507 is the same as S205, and will not be repeated here.

[0150] S508: Based on the above target sub-environment label information and the above target overall environment label information, determine the pseudo-labels corresponding to each of the above target environment image sets, and add the target environment image sets with the above pseudo-labels to the preset labeled training set to obtain the target environment image training set.

[0151] Specifically, S508 is the same as S206, and will not be repeated here.

[0152] S509: Construct a preset prompt word template, which is used to instruct the preset multimodal model to predict labels based on the target environment image set.

[0153] The aforementioned preset prompt word template can refer to an input instruction template used to guide a preset multimodal model to perform recognition output. Optionally, the aforementioned preset prompt word template may include at least one of task description information, label constraint information, and output format constraint information, used to limit the output range and output format of the model. The label constraint information may include multiple reference labels.

[0154] It is understood that the aforementioned reference labels can refer to a pre-defined set of standard label candidates, used as the range of candidate labels for selection or prediction by the multimodal model. These reference labels can include overall environment label categories and sub-environment label categories. The overall environment label category can be used to characterize the overall environment category of the target object, while the sub-environment label categories can be used to characterize the environmental attributes of the target object under different environmental dimensions. For example, when the target object is a store, the overall environment label category can include multiple store environment categories such as first-type stores and second-type stores; the sub-environment label categories corresponding to different environmental dimensions can each include multiple pre-defined environmental attribute labels. The specific overall environment label categories, sub-environment label categories, and their corresponding relationships can be pre-defined according to actual environmental recognition needs. The output format constraint information can be structured output format constraints, used to limit the aforementioned pre-defined multimodal model to output recognition results according to pre-defined fields, pre-defined data structures, or pre-defined text formats, so that the recognition results can be subsequently parsed, compared, and training loss calculated.

[0155] Specifically, task description information can be generated based on the target recognition task, and label constraint information can be generated based on the above-mentioned reference label set. The model output can be limited by combining the preset output format to obtain the above-mentioned preset prompt word template, such as structured fields or lightweight data exchange format (JavaScript Object Notation, JSON) format.

[0156] Optionally, the aforementioned predicted label may refer to the label result output by the preset multimodal model based on the target environment image set and prompt word template, which is used to characterize the model's recognition output result of the target object.

[0157] S510: Input the target environment image sets and the preset prompt word templates from the target environment image training set into the preset multimodal model to obtain the corresponding recognition output results.

[0158] The recognition output result can refer to the output content of the preset multimodal model. The recognition output result can include the predicted label and the corresponding structured output information. The predicted label can be in the form of a structured string, such as a label result in JSON format.

[0159] Specifically, the target environment image set and the prompt word template can be concatenated or encoded based on a preset input format to obtain model input data. This model input data is then input into a preset multimodal model for inference processing, thereby obtaining the corresponding recognition output result. The recognition output result may include the label information predicted by the model and the corresponding structured output content.

[0160] For example, in a food delivery scenario, a prompt word template can be constructed as follows: You are a store environment recognition and review expert. Based on the provided multiple store environment images, please determine the overall environment label matching the store environment and the sub-environment labels corresponding to different environment dimensions from the preset reference labels, and output the recognition results according to the preset structured format. The output format can be limited to JSON structure, and the output fields can be specified to include corresponding fields for carrying the overall environment label and sub-environment labels. Under the action of the above prompt word template, the preset multimodal model can perform inference based on multiple target object environment images in the input target environment image set, determine the overall environment label and one or more sub-environment labels within the preset reference label range, and output the recognition results in a structured form. For example, the model can output the result as: {"result":{"overall_label":"First type of store","sub_labels":["First sub-environment label","Second sub-environment label","Third sub-environment label"]}} where overall_label corresponds to the overall environment label, and sub_labels correspond to one or more sub-environment labels. The field names and structures described above are merely examples. In other implementations, other structured output formats that can distinguish between overall environment labels and sub-environment labels may also be used.

[0161] S511: Based on the above recognition output results and the training label information corresponding to each target environment image set, determine the supervised fine-tuning training loss.

[0162] The training label information refers to the supervised labels corresponding to each target environment image set in the target environment image training set during supervised fine-tuning training. For the original target environment image sets in the pre-defined labeled training set, the training label information can be the corresponding original label information; for the target environment image sets that generate pseudo-labels based on the aforementioned processing and add them to the pre-defined labeled training set, the training label information can be the corresponding pseudo-labels. The aforementioned training label information can include overall environment label information and one or more sub-item environment label information.

[0163] Understandably, the aforementioned supervised fine-tuning training loss can be used to measure the difference between the recognition output and the corresponding training label information. Specifically, the difference between the overall environment label prediction result in the recognition output and the overall environment label in the training label information, as well as the difference between one or more individual environment label prediction results and their corresponding individual environment labels, can be determined based on these differences. Optionally, the loss corresponding to the overall environment label and the loss corresponding to each individual environment label can be calculated separately, and the multiple losses can be fused to obtain the supervised fine-tuning training loss.

[0164] Specifically, the supervised fine-tuning training loss can be determined based on the difference between the predicted labels in the structured string format and the standardized structured label strings corresponding to the training label information. For example, the recognition output can be compared field-by-field with the standardized structured label strings, and the corresponding loss value can be calculated for inconsistent fields. For instance, the loss can be calculated based on the cross-entropy loss for the classification field, or based on the sequence generation loss for the overall matching calculation of the structured strings, thus obtaining the supervised fine-tuning training loss. Furthermore, when multiple sub-labels are involved, the difference between the predicted results corresponding to each sub-label and the target label can be calculated separately, and the losses of each sub-label can be weighted and fused to obtain the overall training loss. For example, the losses for the geographical location label, dining conditions label, and decoration environment label can be calculated separately, and then weighted and summed according to preset weights, so that the key labels have a greater impact on model training. Thus, while improving the consistency of the structured output, fine-grained supervision of multi-dimensional label information can be achieved, thereby improving the prediction accuracy and output stability of the preset multimodal model in the target recognition task.

[0165] S512: Based on the above-mentioned supervised fine-tuning training loss, update the parameters of the above-mentioned preset multimodal model to obtain the target environment recognition model.

[0166] It is understandable that the aforementioned target environment recognition model can refer to a model obtained by supervised fine-tuning training of a preset multimodal model, which has the ability to output target label information based on a target environment image set.

[0167] Specifically, backpropagation can be performed based on the training loss described above to update the model parameters, thereby enabling the model to gradually learn the mapping relationship between input images and prompt words, as well as the corresponding label output patterns. Through multiple rounds of iterative training, the aforementioned pre-defined multimodal model can be adapted to the target recognition task, resulting in the aforementioned target environment recognition model. For example, in the food delivery scenario, by continuously inputting multiple batches of store images and corresponding labels for training, the model can gradually learn the mapping relationship between different store environment features and labels, thereby improving recognition accuracy.

[0168] In this embodiment, by acquiring a set of original environmental images corresponding to multiple target objects and performing image preprocessing and structured sorting on these original environmental image sets, the consistency and integrity of the input data can be effectively improved, thereby enhancing the model's comprehensive understanding of multi-view image information. Furthermore, by introducing multiple recognition models to recognize the target environmental image set and generate pseudo-labels containing target sub-environmental label information and overall label information, automatic annotation of multi-dimensional features of the target objects is achieved, reducing the dependence on manually labeled data and increasing the scale of training data. In addition, by constructing a preset prompt word template, the target recognition task is transformed into a structured label generation task within a constrained label space, and the multimodal model is supervised and fine-tuned based on the prompt word template, enabling the model to learn the ability to generate structured label results from multiple image information under prompt word constraints, thereby improving the consistency and generalization ability of the model output.

[0169] The following combination Figure 6 The training of the aforementioned target environment recognition model is explained. The training process of the target environment recognition model can be constructed as a self-guided active learning method. Specifically, unlabeled data, i.e., multiple target environment image sets corresponding to the aforementioned multiple target objects, can be obtained first. Then, multiple recognition models are used to perform prediction processing on the unlabeled data to obtain multiple candidate environment label information. Finally, the multiple candidate environment label information is evaluated based on a preset evaluation dimension to obtain label evaluation results. Figure 6The "multi-model voting" in this context corresponds to the process of filtering and determining candidate environment label information based on the recognition results of multiple recognition models and the label evaluation results. Based on preset label filtering conditions and the label evaluation results, the target environment image set in unlabeled data can be sampled. Specifically, target environment image sets that meet the preset label filtering conditions are identified as consistent samples; for target environment image sets that do not meet the preset label filtering conditions, label difference information is further determined, and when preset verification conditions are met, they are identified as easily confused samples. Consistent samples can be directly used as high-confidence pseudo-label samples for subsequent training. Specifically, pseudo-labels can be generated based on the target sub-environment label information and the overall target environment label information, and consistent samples with these pseudo-labels are used for subsequent supervised fine-tuning training. Easily confused samples can be further verified as suspected low-confidence samples. For example, corresponding verification sub-environment label information can be determined through manual annotation, and training samples can be constructed based on the verified label information for subsequent supervised fine-tuning training.

[0170] Furthermore, after completing the aforementioned sample selection and labeling, consistent samples and manually corrected easily confused samples can be combined to form training data, resulting in a target environment image training set. This set is then input into a pre-defined multimodal model for supervised fine-tuning training, thereby obtaining an updated target environment recognition model. Based on the updated model, predictions can be made again for unlabeled or historical data, resulting in new sample segmentation results, which can then proceed to the next round of training iterations.

[0171] In some embodiments, after each round of supervised fine-tuning training, the model performance can be evaluated based on a preset validation set, and the performance improvement rate between adjacent training rounds can be calculated. When the performance improvement rate is greater than a preset threshold, the self-guided training process continues; when the performance improvement rate is less than a preset threshold (e.g., less than 1%), the model performance can be determined to have stabilized, and the training iteration process is automatically terminated. Thus, the model can perform sample selection and data optimization based on its own prediction results, combined with manual annotation for error correction, enabling the model to continuously improve recognition accuracy and stability with limited manual costs.

[0172] In one embodiment, such as Figure 7 As shown, a target environment recognition model inference method is provided and applied to e-commerce scenarios. The method includes the following steps: S701: Obtain the target merchant's environmental image set.

[0173] The target merchant environment image set may include multiple images of the merchant object arranged in a preset order, which are used to reflect the external and / or internal environmental features of the store corresponding to the merchant object from multiple perspectives.

[0174] Optionally, the aforementioned target merchant environmental image set may include storefront images, lobby images, dining area images, product display images, and other images reflecting the store's spatial layout or environmental conditions. This target merchant environmental image set may originate from store photo albums uploaded by merchants, images collected by the platform, or third-party data sources, thereby comprehensively characterizing the store's environmental features.

[0175] S702: Input the above target merchant environment image set into the target environment recognition model to obtain the target object environment recognition result output by the above target environment recognition model. The above target object environment recognition result is used to characterize the environment category and corresponding sub-environment attributes of the store object corresponding to the above target merchant.

[0176] The aforementioned target environment recognition model can be a multimodal model trained based on the aforementioned embodiments, which can output structured environment label results based on the input image information.

[0177] Optionally, the above target object environment identification results may include overall target environment label information and multiple target sub-item environment label information, wherein the overall target environment label information is used to characterize the overall environment category of the store, and the target sub-item environment label information is used to characterize the specific attribute characteristics of the store in different dimensions.

[0178] Specifically, the target merchant environment image set can be input into the target environment recognition model according to a preset input format. The target environment recognition model performs feature extraction and semantic understanding on the image information and outputs the corresponding environment recognition result within a preset label space. The target environment recognition result can be represented in a structured form, such as JSON format or a field-based data structure. The target environment recognition model can adopt... Figures 2-5 The target environment recognition model was trained using the method shown.

[0179] In some embodiments, target merchants can be categorized and applied to different service scenarios based on the environment identification results output by the aforementioned target environment identification model. Specifically, the environment category corresponding to the target merchant can be determined based on the overall target environment label information and the target sub-item environment label information in the aforementioned target environment identification results, and a corresponding processing strategy can be determined based on the environment category. For example, corresponding information display methods, data management methods, and / or recommendation processing methods can be adopted for target merchants of different environment categories, so that target merchants with different environment characteristics can adapt to the corresponding service scenarios. Thus, automatic classification and differentiated processing of target merchants can be achieved based on the target object environment identification results, improving the accuracy and efficiency of merchant environment information processing, thereby enhancing the overall content quality and user experience of the platform.

[0180] This application embodiment acquires multi-image information corresponding to the target merchant and identifies the merchant's store environment based on a trained multimodal target environment recognition model. This yields target environment recognition results including overall target environment label information and specific target environment label information, thus comprehensively characterizing the merchant's environment features from multiple dimensions and effectively improving the reliability and comprehensiveness of merchant store environment recognition. Furthermore, merchants can be classified based on the aforementioned target environment recognition results to achieve automatic filtering and risk interception of merchants, as well as automatic identification and priority support for high-quality merchants, thereby reducing manual review costs and improving processing efficiency. In addition, by applying the recognition results to service scenarios such as recommendation ranking, content display, and merchant management, the consistency of the platform's displayed content quality and the stability of the user experience can be effectively improved.

[0181] In some embodiments, a test set including multiple test samples can be constructed, and models obtained by different training methods can be evaluated based on preset evaluation metrics to verify the effectiveness of the above-mentioned target environment recognition model training method. The preset evaluation metrics may include environmental label recognition accuracy and target category recognition rate, etc. Environmental label recognition accuracy can be used to characterize the model's ability to correctly identify multiple preset environmental categories, and target category recognition rate can be used to characterize the model's ability to identify a specific preset environmental category. Test results show that, compared with models directly trained using original training data and other preset benchmark models, the target environment recognition model obtained using the training method provided in this application embodiment has significant improvements in the above-mentioned multiple evaluation metrics. Especially after sample screening, verification processing, and training sample re-entry, the model's overall recognition accuracy for environmental labels and its ability to identify specific environmental categories are further improved. For example, in a set of test results, the initial training model obtained by supervised fine-tuning training using manually labeled data achieved an environment label recognition accuracy of approximately 78% and a target category recognition accuracy of approximately 90.3%. Further employing the target environment recognition model training method provided in this application, through sample selection, verification processing, and training sample re-training, the environment label recognition accuracy of the obtained target environment recognition model can be improved to approximately 93.5%, and the target category recognition accuracy can be improved to approximately 97%, representing improvements of approximately 15.5 percentage points and approximately 6.7 percentage points, respectively, compared to the initial training model. Furthermore, the environment label recognition accuracy of other preset benchmark models is in the range of approximately 65% ​​to 69%, and the target category recognition accuracy is in the range of approximately 83% to 86%. This demonstrates that the target environment recognition model training method provided in this application can effectively improve the model's ability to recognize environment labels and target environment categories.

[0182] Furthermore, during the model iterative training process, the updated target environment recognition model can be used to re-identify the training samples and / or unlabeled environment images. Suspected erroneous samples are then screened based on the consistency of the inference results. After verification, these suspected erroneous samples are re-added to the target environment image training set. Actual testing shows that after performing the above-mentioned re-identification, label comparison, and abnormal sample screening processes using the target environment recognition model training method provided in this application embodiment, the proportion of samples with confirmed label anomalies among the screened suspected erroneous samples can reach over 50%. This indicates that the above processing method can effectively locate potential erroneous labels from the target environment image training set. By verifying and correcting the labeled erroneous samples and reusing them for model training, the impact of erroneous labels on model training can be reduced, thereby continuously improving the label quality of the target environment image training set and the recognition accuracy and stability of the target environment recognition model.

[0183] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0184] Please refer to the following. Figure 8 This is a schematic diagram of the structure of a target environment recognition model training device provided in an exemplary embodiment of this application. Figure 8 As shown, the target environment recognition model training device 800 includes: The first acquisition module 801 is used to acquire multiple target environment image sets corresponding to multiple target objects, wherein the target environment image set corresponding to each of the multiple target objects includes multiple target object environment images acquired from multiple perspectives; The recognition module 802 is used to perform feature recognition on each of the above target environment image sets through multiple preset recognition models to obtain multiple candidate environment label information corresponding to each of the above target environment image sets, wherein the multiple recognition models have different model configurations. Evaluation module 803 is used to evaluate multiple candidate environment label information corresponding to each of the above target environment image sets based on a preset evaluation dimension, and obtain the corresponding label evaluation result. The preset evaluation dimension includes at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. The first determining module 804 is used to perform label determining processing on the above multiple candidate environment label information based on the above label evaluation results, so as to obtain the target sub-item environment label information corresponding to each target environment image set. The association module 805 is used to perform association processing on the above target sub-item environment label information based on a preset hierarchical environment label tree to obtain the overall target environment label information corresponding to each target environment image set. The second determining module 806 is used to determine the pseudo-labels corresponding to each of the target environment image sets based on the target sub-item environment label information and the target overall environment label information, and to add the target environment image sets with the pseudo-labels to a preset labeled training set to obtain the target environment image training set. The training module 807 is used to perform supervised fine-tuning training of the preset multimodal model based on the above target environment image training set to obtain the target environment recognition model.

[0185] In one possible implementation, the aforementioned preset evaluation dimensions also include at least one of the following: label combination consistency, image quality, and historical data stability; When the above-mentioned preset evaluation dimensions include the above-mentioned model consistency, the above-mentioned label evaluation results include the model consistency scores corresponding to the above-mentioned candidate environment label information. When the aforementioned preset evaluation dimensions include the aforementioned cross-graph evidence consistency, the aforementioned label evaluation results include the cross-graph evidence consistency scores corresponding to the aforementioned candidate environment label information. When the above-mentioned preset evaluation dimensions include the above-mentioned label combination consistency, the above-mentioned label evaluation results include the label combination consistency scores corresponding to the above-mentioned candidate environment label information; When the above-mentioned preset evaluation dimensions include the above-mentioned image quality, the above-mentioned label evaluation results include the image quality scores corresponding to each of the above-mentioned candidate environment label information; Given that the aforementioned preset evaluation dimensions include the aforementioned historical data stability, the aforementioned label evaluation results include the historical data stability scores corresponding to the aforementioned candidate environment label information.

[0186] In one possible implementation, the first determining module 804 described above is specifically used for: Based on the preset label screening conditions and the above label evaluation results, the above multiple candidate environmental label information is screened. The candidate environmental label information that meets the above preset label screening conditions is determined as the first type of candidate environmental label information, and the candidate environmental label information that does not meet the above preset label screening conditions is determined as the second type of candidate environmental label information. Based on the first type of candidate environment label information mentioned above, the target sub-item environment label information of the corresponding target environment image set is determined; If the second type of candidate environment label information exists among the multiple candidate environment label information corresponding to the above target environment image set, the label difference information corresponding to the above target environment image set is determined based on the multiple candidate environment label information corresponding to the above target environment image set. If the above label difference information meets the preset review conditions, the above target environment image set is determined as the image set to be reviewed, and the review sub-item environment label information corresponding to the above image set to be reviewed is obtained. Based on the above-mentioned environmental label information for the review items, the target environmental label information corresponding to the above-mentioned image set to be reviewed is determined.

[0187] In one possible implementation, the first determining module 804 described above is specifically used for: The candidate environment label information corresponding to the above target environment image set is standardized to obtain multiple standard candidate environment label information; Determine the number of recognition models corresponding to each of the above multiple standard candidate environment label information, and determine the target standard candidate environment label information with the largest number of corresponding recognition models; Based on the number of recognition models corresponding to the above target standard candidate environment label information and the total number of the above multiple recognition models, the model divergence degree corresponding to the above target environment image set is determined, and the above model divergence degree is determined as label difference information; Under the condition that the aforementioned label difference information meets the preset verification conditions, the aforementioned target environment image set is determined as the image set to be verified, including: If the discrepancy of the above model is greater than or equal to the preset discrepancy threshold, the above target environment image set is determined as the above image set to be reviewed.

[0188] In one possible implementation, the aforementioned associated module 805 is specifically used for: Based on the preset hierarchical environment label tree, determine the label hierarchy information corresponding to the above target sub-item environment label information; Based on the above label hierarchy information and the relationship between different environment labels in the above preset hierarchical environment label tree, the above target item environment label information is combined to obtain multiple candidate overall environment label information corresponding to each target environment image set. Based on the priority relationship between different environment labels in the above-mentioned preset hierarchical environment label tree, the above-mentioned candidate overall environment label information is filtered to obtain the target overall environment label information corresponding to each target environment image set.

[0189] In one possible implementation, the target environment image set includes multiple target object environment images arranged in a preset order; the first acquisition module 801 is specifically used for: Obtain the original environmental image set corresponding to each target object among multiple target objects; Image preprocessing is performed on multiple original object environment images included in each of the above original environment image sets to obtain multiple target object environment images; The environmental images of the above multiple target objects are sorted according to a preset order to obtain the target environment image set corresponding to each target object.

[0190] In one possible implementation, the training module 807 described above is specifically used for: A preset prompt word template is constructed, which is used to instruct the preset multimodal model to predict labels based on the target environment image set; The target environment image sets in the above target environment image training set and the above preset prompt word template are input into the above preset multimodal model to obtain the corresponding recognition output results; Based on the above recognition output results and the training label information corresponding to each target environment image set, the supervised fine-tuning training loss is determined. Based on the above-mentioned supervised fine-tuning training loss, the parameters of the above-mentioned preset multimodal model are updated to obtain the target environment recognition model.

[0191] Please refer to the following. Figure 9 This is a schematic diagram of the structure of a target environment recognition model inference device provided in an exemplary embodiment of this application. Figure 9 As shown, the target environment recognition model inference device 900 includes: The second acquisition module 901 is used to acquire the target merchant's environmental image set corresponding to the target merchant; The reasoning module 902 is used to input the above target merchant environment image set into the target environment recognition model to obtain the target object environment recognition result output by the above target environment recognition model. The target object environment recognition result is used to characterize the environment category and corresponding sub-environment attributes of the store object corresponding to the above target merchant. The aforementioned target environment recognition model is a recognition model generated by training the aforementioned target environment recognition model using the aforementioned target environment recognition model training method.

[0192] This application embodiment also provides an electronic device 1000, which can be a server, and its internal structure diagram can be as follows. Figure 10 As shown. Please see below. Figure 10 The electronic device 1000 includes a processor 1100, a memory 1200, an input / output interface (I / O) 1300, and a communication interface 1400. The processor 1100, memory 1200, and I / O interface 1300 are connected via a system bus 1500, and the communication interface 1400 is connected to the system bus 1500 via the I / O interface 1300. The processor 1100 of the electronic device 1000 provides computing and control capabilities. The memory 1200 of the electronic device 1000 includes a non-volatile storage medium 1210 and internal memory 1220. The non-volatile storage medium 1210 stores an operating system 1211, computer programs 1212, and a database 1213. The internal memory 1220 provides an environment for the operation of the operating system 1211 and computer programs 1212 stored in the non-volatile storage medium 1210. The database 1213 of the electronic device 1000 stores storage service status characteristics, etc. The input / output interface 1300 of the electronic device 1000 is used for exchanging information between the processor 1100 and external devices. The communication interface 1400 of the electronic device 1000 is used for communicating with external terminals via a network connection. The processor 1100 of the electronic device 1000 executes computer program 1212 to implement a target environment recognition model training method or a target environment recognition model inference method.

[0193] This application also provides a computer storage medium storing instructions that, when run on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the above-described computer-readable storage medium.

[0194] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted through the computer storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0195] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0196] The above embodiments are merely descriptions of preferred embodiments of this application and are not intended to limit the scope of this application. Any modifications and improvements made by those skilled in the art to the technical solutions of this application without departing from the spirit of this application should fall within the protection scope defined by the claims.

[0197] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0198] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the features, data, and information involved in this specification were all obtained under full authorization.

Claims

1. A method for training a target environment recognition model, characterized in that, include: Obtain multiple target environment image sets corresponding to multiple target objects, wherein each target object in the multiple target objects' target environment image set includes multiple target object environment images obtained from multiple perspectives; By performing feature recognition on each target environment image set using multiple preset recognition models, multiple candidate environment label information corresponding to each target environment image set is obtained, wherein the multiple recognition models have different model configurations; Based on a preset evaluation dimension, the multiple candidate environment label information corresponding to each target environment image set is evaluated and processed to obtain the corresponding label evaluation result. The preset evaluation dimension includes at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. Based on the label evaluation results, the multiple candidate environment label information is processed to determine the label, thereby obtaining the target sub-environment label information corresponding to each target environment image set; Based on a preset hierarchical environment label tree, the target sub-environment label information is associated to obtain the overall target environment label information corresponding to each target environment image set; Based on the target sub-environment label information and the target overall environment label information, determine the pseudo-labels corresponding to each target environment image set, and add the target environment image sets with the pseudo-labels to a preset labeled training set to obtain the target environment image training set. Based on the target environment image training set, a pre-defined multimodal model is trained under supervision to obtain a target environment recognition model.

2. The method as described in claim 1, characterized in that, The preset evaluation dimensions also include at least one of the following: label combination consistency, image quality, and historical data stability; When the preset evaluation dimension includes the model consistency, the label evaluation result includes the model consistency score corresponding to each candidate environment label information; When the preset evaluation dimension includes the cross-graph evidence consistency, the label evaluation result includes the cross-graph evidence consistency score corresponding to each candidate environment label information; When the preset evaluation dimension includes the label combination consistency, the label evaluation result includes the label combination consistency score corresponding to each candidate environment label information; When the preset evaluation dimension includes the image quality, the label evaluation result includes the image quality score corresponding to each candidate environment label information; When the preset evaluation dimension includes the historical data stability, the label evaluation result includes the historical data stability score corresponding to each candidate environment label information.

3. The method as described in claim 1, characterized in that, The label determination process based on the label evaluation results of the multiple candidate environment label information is performed to obtain the target sub-environment label information corresponding to each target environment image set, including: Based on the preset label filtering conditions and the label evaluation results, the multiple candidate environmental label information is filtered, and the candidate environmental label information that meets the preset label filtering conditions is determined as the first type of candidate environmental label information, and the candidate environmental label information that does not meet the preset label filtering conditions is determined as the second type of candidate environmental label information. Based on the first type of candidate environment label information, the target sub-item environment label information of the corresponding target environment image set is determined; If the second type of candidate environment label information exists among the multiple candidate environment label information corresponding to the target environment image set, the label difference information corresponding to the target environment image set is determined based on the multiple candidate environment label information corresponding to the target environment image set. If the label difference information meets the preset review conditions, the target environment image set is determined as the image set to be reviewed, and the review sub-environment label information corresponding to the image set to be reviewed is obtained; Based on the environmental label information of the review items, the target environmental label information corresponding to the image set to be reviewed is determined.

4. The method as described in claim 3, characterized in that, The step of determining the label difference information corresponding to the target environment image set based on multiple candidate environment label information includes: The multiple candidate environment label information corresponding to the target environment image set is standardized to obtain multiple standard candidate environment label information; Determine the number of recognition models corresponding to each standard candidate environment label information among the multiple standard candidate environment label information, and determine the target standard candidate environment label information with the largest number of corresponding recognition models; Based on the number of recognition models corresponding to the target standard candidate environment label information and the total number of the multiple recognition models, the model divergence degree corresponding to the target environment image set is determined, and the model divergence degree is determined as label difference information; The step of determining the target environment image set as the image set to be reviewed when the label difference information meets the preset review conditions includes: If the model divergence is greater than or equal to a preset divergence threshold, the target environment image set is determined as the image set to be reviewed.

5. The method as described in claim 1, characterized in that, The step of associating the target item environment label information based on a preset hierarchical environment label tree to obtain the overall target environment label information corresponding to each target environment image set includes: Based on a preset hierarchical environment tag tree, determine the tag level information corresponding to the target sub-item environment tag information; Based on the label hierarchy information and the relationship between different environment labels in the preset hierarchical environment label tree, the target sub-item environment label information is combined to obtain multiple candidate overall environment label information corresponding to each target environment image set. Based on the priority relationship between different environment labels in the preset hierarchical environment label tree, the multiple candidate overall environment label information is filtered to obtain the target overall environment label information corresponding to each target environment image set.

6. The method as described in claim 1, characterized in that, The target environment image set includes multiple target object environment images arranged in a preset order; The step of obtaining multiple target environment image sets corresponding to multiple target objects includes: Obtain the original environmental image set corresponding to each target object among multiple target objects; Image preprocessing is performed on multiple original object environment images included in each of the original environment image sets to obtain multiple target object environment images; The multiple target object environment images are sorted according to a preset order to obtain a target environment image set corresponding to each target object.

7. The method as described in claim 1, characterized in that, The step of supervising and fine-tuning the preset multimodal model based on the target environment image training set to obtain the target environment recognition model includes: A preset prompt word template is constructed, which is used to instruct the preset multimodal model to perform label prediction based on the target environment image set; The target environment image sets in the target environment image training set and the preset prompt word template are input into the preset multimodal model to obtain the corresponding recognition output results; Based on the recognition output results and the training label information corresponding to each target environment image set, the supervised fine-tuning training loss is determined. The parameters of the preset multimodal model are updated based on the supervised fine-tuning training loss to obtain the target environment recognition model.

8. A reasoning method for a target environment recognition model, characterized in that, include: Obtain the target merchant's environmental image set; The target merchant's environmental image set is input into the target environment recognition model to obtain the target object environment recognition result output by the target environment recognition model. The target object environment recognition result is used to characterize the environmental category and corresponding sub-environmental attributes of the store object corresponding to the target merchant. The target environment recognition model is a recognition model generated by training the target environment recognition model according to any one of claims 1 to 7.

9. A target environment recognition model training device, characterized in that, include: The first acquisition module is used to acquire multiple target environment image sets corresponding to multiple target objects, wherein the target environment image set corresponding to each target object includes multiple target object environment images acquired from multiple perspectives; The recognition module is used to perform feature recognition on each target environment image set through multiple preset recognition models to obtain multiple candidate environment label information corresponding to each target environment image set, wherein the multiple recognition models have different model configurations; The evaluation module is used to evaluate multiple candidate environment label information corresponding to each target environment image set based on a preset evaluation dimension to obtain the corresponding label evaluation result. The preset evaluation dimension includes at least model consistency and cross-image evidence consistency. The cross-image evidence consistency is used to characterize the degree of consistency between the recognition results of multiple target object environment images corresponding to the same target object for the same label. The first determining module is used to perform label determining processing on the multiple candidate environment label information based on the label evaluation results, so as to obtain the target sub-item environment label information corresponding to each target environment image set; The association module is used to perform association processing on the target sub-environment label information based on a preset hierarchical environment label tree to obtain the overall target environment label information corresponding to each target environment image set; The second determining module is used to determine the pseudo-labels corresponding to each target environment image set based on the target sub-environment label information and the target overall environment label information, and to add the target environment image set with the pseudo-labels to a preset labeled training set to obtain the target environment image training set. The training module is used to perform supervised fine-tuning training of a preset multimodal model based on the target environment image training set to obtain a target environment recognition model.

10. A target environment recognition model inference device, characterized in that, include: The second acquisition module is used to acquire the target merchant's environmental image set corresponding to the target merchant; The inference module is used to input the target merchant's environmental image set into the target environment recognition model to obtain the target object environment recognition result output by the target environment recognition model. The target object environment recognition result is used to characterize the environmental category and corresponding sub-environmental attributes of the store object corresponding to the target merchant. The target environment recognition model is a recognition model generated by training the target environment recognition model according to any one of claims 1 to 7.

11. An electronic device, characterized in that, include: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-8.

12. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method as described in any one of claims 1-8.

13. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer or processor, it causes the computer or processor to perform the method as described in any one of claims 1-8.

Images