Target detection model training and processing method, vehicle and medium

Abstract

The embodiment of the invention relates to a target detection model training and processing method, a vehicle and a medium, and relates to the technical field of vehicle cabin intelligent detection, and the method comprises the steps: obtaining a cabin image in a vehicle cabin, and inputting the cabin image into a target detection model, obtaining a target user field list and a candidate associated object field list, analyzing the target user field list and the candidate associated object field list, obtaining a candidate associated object of which the association relationship with the target user is a preset association relationship as a target associated object, and obtaining that the target associated object exists in the vehicle cabin when the target user leaves the vehicle cabin. The leaving reminding triggering is used for reminding that the target associated object is left in the vehicle cabin. According to the method, the target association object which is in the preset association relationship with the target user in the vehicle cabin can be accurately identified, and when the target user leaves the vehicle cabin and the target association object is still in the vehicle cabin, the leaving reminding is carried out, so that the target detection accuracy is improved and the effectiveness of the leaving reminding is improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent detection technology for vehicle cabins, and in particular to a target detection model training and processing method, a vehicle, and a medium. Background Technology

[0002] Currently, in some driving scenarios, it is necessary to identify targets such as passengers and items in the vehicle cabin to meet usage requirements.

[0003] In related technologies, target detection based on convolutional neural networks combined with sensor signals is cumbersome and complex, and the detection effect is relatively poor, resulting in the inability to provide timely and effective reminders. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a target detection model training and processing method, a vehicle, and a medium.

[0005] This disclosure provides a method for training an object detection model, characterized in that the method includes: acquiring image samples and prompt text; wherein the image samples have corresponding target format information text; wherein the target format information text includes a preset field list, the preset field list including preset fields and a label attribute value corresponding to each preset field; inputting the image samples and the prompt text into a pre-built language visual model, and processing the image samples based on the prompt text using a visual encoder in the pre-built language visual model to obtain a training field list; wherein the training field list includes training fields and a training attribute value corresponding to each training field; adjusting the model parameters of the visual encoder and keeping other model parameters in the pre-built language visual model unchanged based on each field in the training field list and the preset field list and the attribute values ​​corresponding to the fields, to obtain the object detection model.

[0006] Therefore, by constructing data samples and fine-tuning the pre-built language visual model, the resulting target detection model can quickly achieve fine-grained detection of target users and related objects, thereby improving the accuracy of subsequent target detection and enhancing the effectiveness of legacy reminders.

[0007] Optionally, the step of adjusting the model parameters of the visual encoder and keeping other model parameters in the pre-built language visual model unchanged based on each field in the training field list and the preset field list and the attribute values ​​corresponding to the fields to obtain the object detection model includes: calculating multiple field similarities between the training attribute value corresponding to each training field and the label attribute value corresponding to each preset field; calculating a training loss value based on a preset loss function and the multiple field similarities; adjusting the model parameters of the visual encoder and keeping other model parameters in the pre-built language visual model unchanged based on the training loss value and a preset loss threshold until the training loss value is less than the preset loss threshold to obtain the object detection model.

[0008] Therefore, by determining the loss value based on the similarity between the preset fields and the fields processed by the model, the model parameters are continuously adjusted to obtain the final object detection model. This allows the object detection model to accurately acquire each field and the attribute information of each field, further improving the accuracy of subsequent object detection.

[0009] This disclosure provides a target detection processing method, the method comprising: acquiring a cabin image of a vehicle cabin, and inputting the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list; wherein the target user field list is used to represent target user attribute information and the candidate associated object field list is used to represent candidate associated object attribute information; analyzing the target user field list and the candidate associated object field list to obtain candidate associated objects with a preset association relationship with the target user as target associated objects; and, if the target user leaves the vehicle cabin, acquiring the target associated objects existing in the vehicle cabin and triggering a leave-away reminder; wherein the leave-away reminder is used to remind users that the target associated objects have been left in the vehicle cabin.

[0010] Therefore, by processing cabin images through a target detection model, it is possible to accurately identify target objects in the vehicle cabin that have a pre-defined association with the target user, and to provide a reminder when the target user leaves the vehicle cabin but the target object is still in the vehicle cabin, thereby improving the accuracy of target detection and the effectiveness of the reminder.

[0011] Optionally, the target user field list includes target user fields and target user attribute values ​​corresponding to the user fields; the candidate associated object field list includes candidate associated object fields and candidate associated object attribute values ​​corresponding to the candidate associated object fields; the step of analyzing the target user field list and the candidate associated object field list to obtain candidate associated objects with a preset association relationship with the target user as target associated objects includes: determining target user attribute information based on the target user attribute values ​​corresponding to each target user field in the target user field list; wherein, the target user attribute information includes target user identifier, target user location, target user behavior, and target user image. One or more of the location identifiers; determine the candidate associated object attribute information based on the candidate associated object attribute value corresponding to each candidate associated object field in the candidate associated object field list; wherein, the candidate associated object attribute information includes one or more of the following: candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, candidate associated object brand, and relationship between the candidate associated object and the target user; determine the association relationship between the target user and the candidate associated object based on the target user attribute information and the candidate associated object attribute information, and take the candidate associated object whose association relationship is the preset association relationship as the target associated object.

[0012] Therefore, by using a target detection model to accurately acquire multi-dimensional, fine-grained information within the vehicle cabin, such as user attribute information like target user identifiers, locations, and behaviors, as well as related object attribute information like associated object identifiers, categories, colors, brands, and the relationship between the associated object and the target user, the relationship between the associated object and the target user can be accurately determined. Based on this relationship, target associated objects with a pre-defined association with the target user can be identified, thereby improving the efficiency and effectiveness of target detection.

[0013] Optionally, obtaining the target associated object within the vehicle cabin includes: obtaining a current cabin image within the vehicle cabin, and identifying the current cabin image to obtain an identified object; when it is determined that the target associated object exists within the identified object, it is determined that the target associated object exists within the vehicle cabin.

[0014] Therefore, when the target user leaves the vehicle cabin, the system obtains the identification objects present in the current vehicle cabin through the current cabin image, and determines the presence of the target associated object in the vehicle cabin when it judges that the target associated object exists among the identification objects, thus ensuring the accuracy of the identification and the effectiveness of subsequent residual reminders, and meeting the user's usage needs.

[0015] Optionally, the method further includes: determining target associated object attribute information based on the target associated object field list of the target associated object; wherein the target associated object attribute information includes one or more of the following: target associated object identifier, target associated object category, target associated object image location identifier, target associated object color, target associated object brand, and the relationship between the target associated object and the target user; determining a target reminder method based on the target associated object attribute information, and triggering the legacy reminder based on the target reminder method.

[0016] Therefore, when it is determined that there is a target associated object in the vehicle cabin, the corresponding target reminder method is determined based on one or more target associated object attribute information such as target object identifier, category, color, brand, and the relationship between the target associated object and the target user. The legacy reminder is then triggered according to the target reminder method, thereby enabling different reminder methods to be selected for legacy reminders based on different target associated objects, ensuring that legacy reminders can be accurately received by the user, and further improving the flexibility and effectiveness of legacy reminders.

[0017] Optionally, after obtaining the target associated object in the vehicle cabin, the method further includes: when the door is detected to be closed, triggering the legacy reminder according to a preset first reminder strategy; or, when the door is detected to be open, triggering the legacy reminder according to a preset second reminder strategy.

[0018] Therefore, after obtaining information about the presence of a target-related object in the vehicle's cabin, the system detects the door status and determines the distance between the target user and the vehicle. Based on the vehicle status, different reminder strategies are selected to trigger legacy reminders, ensuring that the target user can effectively receive legacy reminders and guaranteeing their effectiveness.

[0019] Optionally, when the vehicle door is detected to be not closed, the target associated object is an intelligent in-vehicle machine device with motion functions, and the method further includes: sending an action enable command to the target associated object; wherein the action enable command is used to control the target associated object to activate the motion function and leave the vehicle cabin.

[0020] Therefore, when the detection door is not closed and the target associated object is an intelligent in-vehicle machine with motion function, the target associated object can be directly controlled to start the motion function and leave the vehicle cabin, which enriches the interaction between the vehicle and the intelligent in-vehicle device while improving the convenience of control.

[0021] This disclosure also provides an object detection model training apparatus, the apparatus comprising: a first acquisition module, configured to acquire image samples and prompt text; wherein the image samples have corresponding target format information text; wherein the target format information text includes a preset field list, the preset field list including preset fields and label attribute values ​​corresponding to each preset field; an input module, configured to input the image samples and the prompt text into a pre-built language visual model, and process the image samples based on the prompt text using a visual encoder in the pre-built language visual model to obtain a training field list; wherein the training field list includes training fields and training attribute values ​​corresponding to each training field; and a generation module, configured to adjust the model parameters of the visual encoder and control other model parameters in the pre-built language visual model to remain unchanged based on each field in the training field list and the preset field list and the attribute values ​​corresponding to the fields, to obtain the object detection model.

[0022] This disclosure also provides a target detection processing apparatus, comprising: a second acquisition module for acquiring a cabin image inside a vehicle cabin; a processing module for inputting the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list; wherein the target user field list represents target user attribute information and the candidate associated object field list represents candidate associated object attribute information; an analysis module for analyzing based on the target user field list and the candidate associated object field list to obtain candidate associated objects with a preset association relationship with the target user as target associated objects; and a reminder module for, when the target user leaves the vehicle cabin, acquiring the target associated object existing in the vehicle cabin and triggering a leave-away reminder; wherein the leave-away reminder is used to remind the user that the target associated object was left in the vehicle cabin.

[0023] This disclosure also provides a vehicle, the electronic device including: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the target detection model training method and the target detection processing method provided in this disclosure.

[0024] This disclosure also provides a computer-readable storage medium storing a computer program for executing the target detection model training method and target detection processing method provided in this disclosure.

[0025] This disclosure also provides a computer program product, including a computer program, wherein the computer program is executed by a processor as described in the embodiments of this disclosure, including the target detection model training method and the target detection processing method.

[0026] The technical solution provided in this disclosure has the following advantages compared with the prior art: The target detection processing scheme provided in this disclosure includes: acquiring a cabin image inside a vehicle cabin, inputting the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list; analyzing the target user field list and the candidate associated object field list to obtain candidate associated objects with a preset association relationship with the target user as target associated objects; and, when the target user leaves the vehicle cabin, detecting the presence of target associated objects within the vehicle cabin and triggering a leave-away reminder; wherein, the leave-away reminder is used to remind users that target associated objects have been left behind in the vehicle cabin. Therefore, it is possible to accurately identify target associated objects within the vehicle cabin with a preset association relationship with the target user, and to issue a leave-away reminder when the target associated object is still in the vehicle cabin when the target user leaves, thereby improving the accuracy of target detection and the effectiveness of the leave-away reminder.

[0027] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0028] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0029] Figure 1 A flowchart illustrating a target detection model training method provided in this embodiment of the disclosure; Figure 2 A schematic flowchart of a target detection processing method provided in an embodiment of this disclosure; Figure 3 A schematic flowchart of another target detection processing method provided in this embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of a target detection model training device provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of a target detection processing device provided in an embodiment of the present disclosure; Figure 6This is a schematic diagram of the structure of a vehicle provided in an embodiment of the present disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] In real-world applications, many passengers often leave objects behind after getting off the bus, causing inconvenience in the subsequent retrieval process. Existing methods, such as convolutional neural network-based object detection, learn to locate objects in images and classify them from labeled training data through supervised learning. However, they cannot directly judge fine-grained features such as material, color, and brand. They also require the use of other specialized models and sensor signals, along with complex post-processing logic, to achieve partial feature judgment. The entire process is cumbersome and complex, and the overall performance of the model is limited by the quantity and quality of training data, resulting in poor object detection performance and ultimately failing to provide timely and effective reminders for left-behind items.

[0037] To address the aforementioned technical issues, this disclosure proposes a target detection scheme. The scheme involves acquiring an image of the vehicle's cabin and inputting it into a target detection model to obtain a target user field list and a candidate associated object field list. Based on these lists, analysis is performed to identify candidate associated objects with a pre-defined association relationship to the target user, which are then selected as target associated objects. If the target user leaves the vehicle cabin, the presence of a target associated object within the cabin is detected, triggering a leave-behind reminder. This reminder indicates that the target associated object was left behind in the vehicle cabin. Therefore, this scheme can accurately identify target associated objects with a pre-defined association relationship to the target user within the vehicle cabin and provide a leave-behind reminder when the target associated object remains in the cabin after the user leaves, thus improving the accuracy of target detection and the effectiveness of the leave-behind reminder.

[0038] Figure 1 This is a flowchart illustrating a target detection model training method provided in an embodiment of this disclosure. This target detection model training method can be executed by an target detection model training device, which can be implemented in software and / or hardware, and is generally integrated into a device. Figure 1 As shown, it includes: Step 101: Obtain image samples and prompt text; wherein, the image samples have corresponding target format information text; wherein, the target format information text includes a list of preset fields, and the list of preset fields includes preset fields and the label attribute value corresponding to each preset field.

[0039] Step 102: Input the image samples and prompt text into the pre-built language vision model. Based on the visual encoder in the pre-built language vision model, process the image samples based on the prompt text to obtain a training field list. The training field list includes training fields and training attribute values ​​corresponding to each training field.

[0040] Step 103: Based on the training field list and the preset field list, adjust the model parameters of the visual encoder and keep the other model parameters in the pre-built language visual model unchanged to obtain the object detection model.

[0041] Specifically, in order to address the problem that traditional object detection algorithms cannot extract fine-grained features, this embodiment of the present disclosure determines structured information text (such as JSON (JavaScript Object Notation) and other target format information text) that can describe fine-grained object detection based on the cockpit scene, constructs a supervised fine-tuning dataset for the cockpit vertical domain, and fine-tunes the visual encoder parameters of the visual language model. This allows the visual language model to retain its original language capabilities while having a more robust feature extraction capability for cockpit images, thereby outputting fine-grained target detection information for the cockpit.

[0042] For example, the image samples can be multiple images acquired from different angles and object scenes inside the vehicle cabin. The specific selection depends on the application scenario. The target format information text is JSON formatted text containing a list of preset fields, such as "humans," "objects," and "pets." The "humans" field list may include multiple tasks; if no person is detected, this list is empty; if a person is detected, the preset field list includes preset fields such as "human_id," "position," "action," and "bbox." "bbox" represents general detection information, while the other fields represent fine-grained information. "human_id" is a unique identifier for a person, with attribute values ​​such as "human_001," "human_002," etc., which can be sequentially incremented. The attribute value for "position" can be "driver," "copilot," "middle_left," "middle_center," "middle_right," "rear_left," "rear_right," etc. "action"... The corresponding attribute value represents the character's current behavior, and the attribute value corresponding to "bbox" represents the character's bounding box position in the image.

[0043] The "objects" field list represents multiple possible target objects. If no target object is detected, the list is empty. If a target object is detected, the field list includes preset fields such as "object_id", "object_category", "bbox", "color", "material", "brand", and "relation". The "relation" field can include three subfields: "human_id", "seat_id", and "body_contact". The attribute value corresponding to "object_id" represents the unique identifier of the target object; the attribute value corresponding to "object_category" represents the category of the target object; the attribute value corresponding to "bbox" represents the bounding box position of the target object in the image; the attribute value corresponding to "color" represents the color of the target object; the attribute value corresponding to "material" represents the material of the target object; the attribute value corresponding to "brand" represents the brand of the target object; the attribute value corresponding to "relation" describes the relationship between the target object and the person and the seat: "human_id" represents the identifier of the person associated with it; "seat_id" indicates the seat identifier if placed on a seat, otherwise empty; "body_contact" indicates "true" if there is contact with the person's body, otherwise "false", etc.

[0044] The "pets" preset field list represents multiple possible pet objects. If no pet is detected, the list is empty. If a pet is detected, the preset field list includes preset fields such as "pet_id", "pet_category", "bbox", "color", and "relation". "Relation" can include three subfields: "human_id", "seat_id", and "body_contact". The attribute value of "pet_id" represents the unique identifier of the pet, the attribute value of "pet_category" represents the pet category, the attribute value of "bbox" represents the bounding box position of the pet in the image, the attribute value of "color" represents the color of the pet, and the attribute value of "relation" is used to describe the relationship between the target object and the person and the seat. "human_id" represents the identifier of the person with whom it is related, "seat_id" represents the seat identifier if it is placed on the seat, otherwise it is empty, and "body_contact" represents "true" if there is contact with the person's body, otherwise it is "false", etc.

[0045] It should be noted that a list of preset fields can be selected based on the actual application scenario, as well as the preset fields in each preset field list and the corresponding tag attribute values ​​for each preset field. The people mentioned above can be used as target users, and the target objects and pets can be used as candidate associated objects.

[0046] Specifically, based on the basic visual language model, prompt text is designed, target format information text is filled in, and the target format information text is manually corrected based on image samples. Finally, the prompt text, image samples, and target format information text are integrated to construct an SFT (Supervised Fine-Tun-ing) fine-tuning dataset for fine-tuning the visual language model. More specifically, the parameters of the language encoder and multimodal mapping layer of the visual language model are first frozen, and then the parameters of the visual encoder are trained. This can preserve the semantic understanding ability of the visual language model learned from massive text information, while also allowing it to learn the visual features specific to the cockpit domain.

[0047] In this embodiment, the model parameters of the visual encoder are adjusted based on each field in the training field list and the preset field list, as well as the attribute values ​​corresponding to the fields, while keeping other model parameters in the pre-built language visual model unchanged, to obtain an object detection model. This can be understood as either adjusting the model a certain number of times to reach a certain threshold, or calculating the training loss value based on each field in the training field list and the preset field list, as well as the attribute values ​​corresponding to the fields, and continuously adjusting the model parameters of the visual encoder based on the training loss value and the preset loss threshold while keeping other model parameters in the pre-built language visual model unchanged until the training loss value is less than or equal to the preset loss threshold, to obtain an object detection model.

[0048] Specifically, the similarity between the training attribute value corresponding to each training field and the label attribute value corresponding to each preset field is calculated across multiple fields. A training loss value is obtained based on a preset loss function and the multiple field similarities. The model parameters of the visual encoder are adjusted based on the training loss value and a preset loss threshold, while other model parameters in the pre-built language visual model are kept constant until the training loss value is less than the preset loss threshold, thus obtaining the object detection model. Therefore, the large visual language model can not only meet the needs of object localization but also achieve fine-grained extraction of object information. Furthermore, by constructing reasonable data samples and fine-tuning the large visual language model, fine-grained object detection can be quickly achieved.

[0049] Therefore, by constructing data samples and fine-tuning the pre-built language visual model, fine-grained detection of target users and related objects can be quickly achieved, thereby improving the accuracy of subsequent target detection and enhancing the effectiveness of legacy reminders.

[0050] Figure 2This is a flowchart illustrating a target detection processing method provided in an embodiment of the present disclosure. This target detection processing method can be executed by a target detection processing device, which can be implemented using software and / or hardware, and is generally integrated into a device. Figure 2 As shown, this target detection processing method is applied to the device and includes: Step 201: Obtain a cabin image inside the vehicle's cockpit and input the cabin image into the target detection model to obtain a target user field list and a candidate associated object field list; wherein, the target user field list is used to represent the target user attribute information and the candidate associated object field list is used to represent the candidate associated object attribute information.

[0051] The cockpit image refers to the image captured by a vehicle cockpit using a pre-set image acquisition device (such as a camera); there can be one or more pre-set image acquisition devices, and the corresponding cockpit images can also be one or more.

[0052] In this embodiment of the disclosure, the cockpit image is input into the target detection model to obtain a target user field list and a candidate associated object field list; wherein, the target detection model refers to the pre-constructed language visual model obtained by training the image sample, the preset field list corresponding to the image sample, and the prompt text. The specific training process is described in detail in the following embodiments and will not be described in detail here.

[0053] The target user refers to one or more users in the vehicle cabin. The target user field list is used to represent target user attribute information, including one or more of the following: target user identifier, target user location, target user behavior, and target user location identifier in the cabin image. Different target user fields represent different target user attribute information.

[0054] Among them, candidate associated objects refer to one or more objects in the vehicle cabin, such as mobile phones, watches, or living things like pets. The candidate associated object field list is used to represent the attribute information of candidate associated objects, including one or more of the following: candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object brand, and the relationship between the candidate associated object and the target user. Different candidate associated object fields represent different candidate associated object attribute information.

[0055] Specifically, after acquiring the cabin image inside the vehicle, inputting the cabin image into the object detection model can obtain a list of target user fields and a list of candidate associated object fields, thereby obtaining the target user attribute information and candidate associated object attribute information corresponding to the cabin image.

[0056] Step 202: Analyze the target user field list and the candidate associated object field list to obtain the candidate associated objects whose association with the target user is a preset association relationship as the target associated objects.

[0057] In this embodiment of the disclosure, after obtaining the target user field list and the candidate associated object field list, analysis is performed on the target user field list and the candidate associated object field list to accurately obtain the candidate associated objects whose association relationship with the target user is a preset association relationship as the target associated objects.

[0058] It is understandable that the target user field list includes one or more target user fields, each with a corresponding target user attribute value, thereby allowing the acquisition of one or more target user attribute information; among which, the target user attribute information includes one or more of the following: target user identifier, target user location, target user behavior, and target user image location identifier.

[0059] It is understandable that the candidate associated object field list includes one or more candidate associated object fields, each candidate associated object field has a corresponding candidate associated object attribute value, thereby obtaining one or more candidate associated object attribute information; among them, the candidate associated object attribute information includes one or more of the following: candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, candidate associated object brand, and the relationship between the candidate associated object and the target user.

[0060] Furthermore, based on the target user attribute information and the candidate associated object attribute information, the association relationship between the target user and each candidate associated object can be determined. For example, if the target user and the candidate associated object is a mobile phone, the association relationship is that the mobile phone belongs to the target user. Or, if the target user and the candidate associated object is a seat pillow, the association relationship is that the seat pillow and the target user are not associated. Through the analysis of the target user attribute information and the candidate associated object attribute information, the association relationship between all candidate associated objects in the vehicle cabin and the target user can be determined.

[0061] Among them, the preset association relationship refers to the pre-set association relationship, such as the affiliation relationship. After determining the association relationship between all candidate association objects in the vehicle cabin and the target user, it is determined whether the association relationship is a preset association relationship. Candidate association objects with a preset association relationship with the target user are selected as target association objects. In other words, candidate association objects with a preset association relationship with the target user are selected as target association objects.

[0062] Step 203: When the target user leaves the vehicle cabin, obtain information that the target associated object exists in the vehicle cabin and trigger a leave-behind reminder; the leave-behind reminder is used to remind the user that the target associated object has been left in the vehicle cabin.

[0063] In this embodiment of the disclosure, when a target user leaves the vehicle cabin, the presence of a target associated object in the vehicle cabin can be detected by the vehicle's preset sensors, cameras, or operations such as turning off the vehicle, opening the door, and locking the vehicle, triggering a reminder to indicate that the target associated object has been left in the vehicle cabin.

[0064] In the embodiments of this disclosure, there are many ways to obtain the existence of a target associated object in the vehicle cabin. In some embodiments, the current cabin image in the vehicle cabin is obtained and the current cabin image is identified to obtain the identified object. When it is determined that the identified object has a target associated object, the existence of a target associated object in the vehicle cabin is determined. In other embodiments, the target associated object in the vehicle cabin is monitored by a preset sensor to determine that the existence of a target associated object in the vehicle cabin is determined.

[0065] Specifically, when the target user leaves the vehicle cabin, a reminder is sent that the target associated object is left in the vehicle cabin.

[0066] The target detection processing scheme provided in this disclosure acquires an image of the vehicle cabin and inputs it into a target detection model to obtain a target user field list and a candidate associated object field list. Based on the target user field list and the candidate associated object field list, analysis is performed to obtain candidate associated objects with a preset association relationship with the target user as target associated objects. If the target user leaves the vehicle cabin, the presence of a target associated object within the vehicle cabin is detected, triggering a leave-behind reminder. This leave-behind reminder is used to indicate that a target associated object has been left in the vehicle cabin. Therefore, it can accurately identify target associated objects within the vehicle cabin with a preset association relationship with the target user, and provide a leave-behind reminder when the target associated object is still in the vehicle cabin after the target user leaves, thus improving the accuracy of target detection and the effectiveness of the leave-behind reminder.

[0067] In some embodiments, the target user field list includes target user fields and target user attribute values ​​corresponding to the user fields, and the candidate associated object field list includes candidate associated object fields and candidate associated object attribute values ​​corresponding to the candidate associated object fields. Analysis is performed based on the target user field list and the candidate associated object field list to obtain candidate associated objects whose association with the target user is a preset association relationship as target associated objects. This includes: determining target user attribute information based on the target user attribute values ​​corresponding to each target user field in the target user field list; wherein the target user attribute information includes one or more of the following: target user identifier, target user location, target user behavior, and target user image location identifier; determining candidate associated object attribute information based on the candidate associated object attribute values ​​corresponding to each candidate associated object field in the candidate associated object field list; wherein the candidate associated object attribute information includes one or more of the following: candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, candidate associated object brand, and the relationship between the candidate associated object and the target user; determining the association relationship between the target user and the candidate associated objects based on the target user attribute information and the candidate associated object attribute information, and selecting candidate associated objects whose association relationship is a preset association relationship as target associated objects.

[0068] Specifically, the target user field list includes one or more target user fields, each with a corresponding target user attribute value. For example, the target user fields include a target user identifier field, a target user location field, a target user behavior field, and a target user image location identifier field. Correspondingly, each target user field has a corresponding target user attribute value. The attribute value for the target user identifier field is "human_001", the attribute value for the target user location field is "rear_left", the attribute value for the target user behavior field is "holding", and the attribute value for the target user image location identifier field is "bbox", which represents the bounding box position of the target user in the image. Thus, the target user attribute information can be obtained as the target user identifier "human_001", the target user location "rear_left", the target user behavior "holding", and the target user image location identifier "bbox1".

[0069] Among them, the target user identifier is used to uniquely identify a target user; the target user location is used to indicate the target user's position in the vehicle cabin, such as the driver, the left rear seat, etc.; the target user behavior is used to indicate the target user's current behavior when the cabin image is acquired; and the target user image location identifier is used to indicate the target user's bounding box position in the cabin image.

[0070] Specifically, the candidate associated object field list includes one or more candidate associated object fields, each with a corresponding candidate associated object attribute value. For example, the candidate associated object fields include a candidate associated object identifier field, a candidate associated object category field, a candidate associated object image location identifier field, a candidate associated object color field, a candidate associated object brand field, and a candidate associated object-to-target-user relationship field. Correspondingly, each candidate associated object field has a corresponding candidate associated object attribute value. For instance, the candidate associated object identifier field has the attribute value "object_001", the candidate associated object category field has the attribute value "category1", the candidate associated object image location identifier field has the attribute value "bbox2", and so on. Select the attribute value corresponding to the color field of the candidate associated object as "color1", the material field of the candidate associated object as "material1", the brand field of the candidate associated object as "brand1", and the relationship field between the candidate associated object and the target user as "relation1". In this way, the candidate associated attribute information can be obtained as the candidate associated object identifier "object_001", the candidate associated object category "category1", the candidate associated object image location identifier "bbox2", the candidate associated object color "color1", the candidate associated object material "material1", the candidate associated object brand "brand1", and the relationship between the candidate associated object and the target user "relation1".

[0071] Among them, the candidate associated object identifier is used to uniquely identify a candidate associated object; the candidate associated object location is used to indicate the position of the candidate associated object in the vehicle cabin, such as in the target user's hand or on the seat; the candidate associated object behavior is used to indicate the current behavior of the candidate associated object when the cabin image is acquired; the candidate associated object image location identifier is used to indicate the bounding box position of the candidate associated object in the cabin image; the candidate associated object category is used to indicate the category to which the candidate associated object belongs, such as electronic devices or bags; the candidate associated object color is used to indicate the color of the candidate associated object, such as green or red; the candidate associated object material is used to indicate the material of the candidate associated object, such as metal or plastic; the candidate associated object brand is used to indicate the brand of the candidate associated object; and the candidate associated object-target user relationship is used to indicate the relationship between the candidate associated object and the target user, such as having contact with the target user.

[0072] Furthermore, the association relationship between the target user and the candidate associated object is determined based on the target user attribute information and the candidate associated object attribute information, and the candidate associated object with the preset association relationship is taken as the target associated object.

[0073] Therefore, by using a target detection model to accurately acquire multi-dimensional, fine-grained information within the vehicle cabin, such as user attribute information like target user identifiers, locations, and behaviors, as well as related object attribute information like associated object identifiers, categories, colors, brands, and the relationship between the associated object and the target user, the relationship between the associated object and the target user can be accurately determined. Based on this relationship, target associated objects with a pre-defined association with the target user can be identified, thereby improving the efficiency and effectiveness of target detection.

[0074] In some embodiments, obtaining the existence of a target associated object in the vehicle cabin includes: obtaining a current cabin image in the vehicle cabin, recognizing the current cabin image to obtain a recognized object, and determining that a target associated object exists in the vehicle cabin when it is determined that the recognized object has a target associated object.

[0075] Specifically, after the target user leaves the vehicle cabin, the current cabin image can be acquired again through image acquisition equipment. The current cabin image can then be identified using image recognition algorithms or image recognition models, such as the aforementioned target detection model, to obtain the identified object. The identified object refers to the object remaining in the vehicle cabin after the target user leaves, such as items like mobile phones or bags, or even living creatures like pets.

[0076] Furthermore, after acquiring the identification object, it is determined whether the identification object contains a target associated object. If it is determined that the identification object contains a target associated object, it is determined that a target associated object exists in the vehicle cabin.

[0077] Therefore, when the target user leaves the vehicle cabin, the system obtains the identification objects present in the current vehicle cabin through the current cabin image, and determines the presence of the target associated object in the vehicle cabin when it judges that the target associated object exists among the identification objects, thus ensuring the accuracy of the identification and the effectiveness of subsequent residual reminders, and meeting the user's usage needs.

[0078] In some embodiments, the target detection processing method further includes: determining target associated object attribute information based on a target associated object field list; wherein the target associated object attribute information includes one or more of the following: target associated object identifier, target associated object category, target associated object image location identifier, target associated object color, target associated object brand, and the relationship between the target associated object and the target user; determining a target reminder method based on the target associated object attribute information, and triggering a legacy reminder based on the target reminder method.

[0079] In this embodiment of the disclosure, before triggering a legacy reminder by obtaining information about the existence of a target associated object in the vehicle cabin, different target reminder methods can be selected based on different attribute information of the target associated object to trigger the legacy reminder.

[0080] Specifically, one or more of the following can be used as target associated object attribute information: target associated object identifier, target associated object category, target associated object image location identifier, target associated object color, target associated object brand, and the relationship between the target associated object and the target user. For example, if the target associated object is determined to be a pet based on the target associated object attribute information, the target reminder method can be determined to be a combination of flashing car lights, honking the horn, and in-car voice broadcast for a quick and effective prompt; or if the target associated object is determined to be a bag based on the target associated object attribute information, the target reminder method can be determined to be flashing car lights, etc.

[0081] Therefore, when it is determined that there is a target associated object in the vehicle cabin, the corresponding target reminder method is determined based on one or more target associated object attribute information such as target object identifier, category, color, brand, and the relationship between the target associated object and the target user. The legacy reminder is then triggered according to the target reminder method, thereby enabling different reminder methods to be selected for legacy reminders based on different target associated objects, ensuring that legacy reminders can be accurately received by the user, and further improving the flexibility and effectiveness of legacy reminders.

[0082] Optionally, after obtaining the existence of a target-related object in the vehicle cabin, the target detection processing method further includes: triggering a legacy reminder according to a preset first reminder strategy when the door is detected to be closed; or triggering a legacy reminder according to a preset second reminder strategy when the door is detected to be open.

[0083] In this embodiment of the disclosure, after obtaining the existence of a target associated object in the vehicle cabin, the door status can also be detected. The distance and other relationships between the user and the vehicle can be determined through the door status. Thus, when the door is in a closed state, a residual reminder is triggered according to a preset first reminder strategy, such as flashing headlights or honking the horn. If the door is not in a closed state, a residual reminder is triggered according to a preset second reminder strategy, such as displaying reminder information on the in-vehicle display screen or in-vehicle voice broadcast.

[0084] Therefore, after obtaining information about the presence of a target-related object in the vehicle's cabin, the system detects the door status and determines the distance between the target user and the vehicle. Based on the vehicle status, different reminder strategies are selected to trigger legacy reminders, ensuring that the target user can effectively receive legacy reminders and guaranteeing their effectiveness.

[0085] Optionally, when the detection door is not closed, the target associated object is an intelligent in-vehicle machine device with motion function, and the target detection processing method further includes: sending an action enable command to the target associated object; wherein, the action enable command is used to control the target associated object to activate the motion function and leave the vehicle cabin.

[0086] In this embodiment of the disclosure, when the vehicle door is detected to be not closed, the target associated object is an intelligent vehicle-mounted machine device with motion functions, such as a robot dog. An action enable command can be sent to the target associated object to control the target associated object to activate its motion function and leave the vehicle cabin, thereby further improving the flexibility of intelligent device control and meeting user needs.

[0087] Therefore, when the detection door is not closed and the target associated object is an intelligent in-vehicle machine with motion function, the target associated object can be directly controlled to start the motion function and leave the vehicle cabin, which enriches the interaction between the vehicle and the intelligent in-vehicle device while improving the convenience of control.

[0088] Figure 3 This is a flowchart illustrating another target detection processing method provided in this embodiment of the present disclosure. This embodiment further optimizes the above-described target detection processing method based on the previous embodiment. Figure 3 As shown, the method includes: Step 301: Obtain the cabin image inside the vehicle cabin and input the cabin image into the target detection model to obtain the target user field list and the candidate associated object field list.

[0089] Step 302: Determine the target user attribute information based on the target user attribute value corresponding to each target user field in the target user field list; wherein, the target user attribute information includes one or more of the following: target user identifier, target user location, target user behavior, and target user image location identifier.

[0090] Step 303: Determine the attribute information of the candidate associated object based on the attribute value of the candidate associated object corresponding to each candidate associated object field in the candidate associated object field list; wherein, the attribute information of the candidate associated object includes one or more of the following: candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, candidate associated object brand, and the relationship between the candidate associated object and the target user.

[0091] Step 304: Determine the association relationship between the target user and the candidate associated object based on the target user attribute information and the candidate associated object attribute information, and take the candidate associated object with the preset association relationship as the target associated object.

[0092] Step 305: When the target user leaves the vehicle cabin, acquire the current cabin image inside the vehicle cabin, and identify the current cabin image to obtain the identified object. If the identified object contains the target associated object, determine that the target associated object exists inside the vehicle cabin.

[0093] Step 306: Determine the target associated object attribute information based on the target associated object field list, determine the target reminder method based on the target associated object attribute information, and trigger a legacy reminder based on the target reminder method to remind that the target associated object has been left in the vehicle cabin.

[0094] Specifically, after acquiring the cabin image inside the vehicle cabin, inputting the cabin image into the target detection model can obtain a target user field list and a candidate associated object field list, thereby obtaining the target user attribute information and candidate associated object attribute information corresponding to the cabin image. Based on the target user attribute information and candidate associated object attribute information, the association relationship between the target user and the candidate associated object is determined, and the candidate associated object with the preset association relationship is taken as the target associated object.

[0095] Specifically, after the target user leaves the vehicle cabin, the current cabin image can be acquired again through image acquisition equipment. The current cabin image can be identified by image recognition algorithm or image recognition model, or the aforementioned target detection model, to obtain the identified object. It is then determined whether the identified object contains a target-related object. If the identified object contains a target-related object, it is determined that a target-related object exists in the vehicle cabin.

[0096] Furthermore, before triggering a legacy reminder, different target reminder methods can be selected based on the attribute information of different target associated objects, after obtaining the existence of target associated objects in the vehicle cabin.

[0097] For example, the output target user field list contains non-empty elements, where "human_id" is "human_001" and "action" is "holding"; the output candidate associated object field list contains non-empty elements, where "object_id" is "object_001", and the corresponding "relation" field contains "human_id" as "human_001". Based on the corresponding information, the interaction relationship between the person with "human_id" "human_001" and the object with "object_id" "object_001" can be deduced; if the content of the "action" field is "calling", it can be deduced that the corresponding "human_001" is using "object_001" to make a phone call, thus determining that the association relationship between "human_001" and "object_001" is a belonging relationship that satisfies the preset association relationship, and thus "object_001" is taken as the target associated object.

[0098] Therefore, by using object detection models to achieve more intelligent and flexible cabin status analysis, it is possible to accurately capture multi-dimensional fine-grained information within the vehicle cabin. It can not only analyze information such as people, objects, and pets in images, but also more subtly determine the interaction information between people and objects / pets, transforming fragmented visual information into clearly structured fine-grained information. In addition, the cross-modal understanding capability of object detection models makes information extraction more aligned with actual needs, supports the definition of extraction rules in natural language, filters redundant information by combining cabin scene knowledge, and outputs structured fine-grained data. This provides robust underlying support for intelligent interaction and safety warnings, transforming cabin perception from seeing to understanding, and further improving the accuracy of object detection to enhance the effectiveness of residual reminders.

[0099] The target detection processing scheme provided in this disclosure acquires a cabin image inside a vehicle's cockpit and inputs the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list. Target user attribute information is determined based on the target user attribute values ​​corresponding to each target user field in the target user field list. The target user attribute information includes one or more of the following: target user identifier, target user location, target user behavior, and target user image location identifier. Candidate associated object attribute information is determined based on the candidate associated object attribute values ​​corresponding to each candidate associated object field in the candidate associated object field list. The candidate associated object attribute information includes the candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, and candidate associated object type. The system selects one or more of the following relationships: brand of associated objects and candidate associated objects with the target user. Based on the target user's attribute information and the attribute information of the candidate associated objects, it determines the association between the target user and the candidate associated objects. Candidate associated objects with a preset association relationship are designated as target associated objects. When the target user leaves the vehicle cabin, the system acquires an image of the current cabin and identifies the identified objects. If the identified objects include the target associated object, it confirms the presence of the target associated object within the vehicle cabin. Based on the target associated object's field list, it determines the attribute information of the target associated object. Based on the target associated object's attribute information, it determines the target reminder method and triggers a leave-in reminder to alert the user that the target associated object was left in the vehicle cabin. This system can accurately identify target associated objects within the vehicle cabin with a preset association relationship with the target user and provides a leave-in reminder when the target associated object is still in the vehicle cabin after the user leaves, improving target detection accuracy and thus the effectiveness of the leave-in reminder.

[0100] Figure 4 This is a schematic diagram of a target detection model training device provided in an embodiment of this disclosure. The device can be implemented by software and / or hardware, and is generally integrated into a device. Figure 4 As shown, the device includes: The first acquisition module 410 is used to acquire image samples and prompt text; wherein the image samples have corresponding target format information text; wherein the target format information text includes a preset field list, the preset field list includes preset fields and a tag attribute value corresponding to each preset field; The input module 420 is used to input the image sample and the prompt text into a pre-built language vision model, and to process the image sample based on the prompt text using the visual encoder in the pre-built language vision model to obtain a training field list; wherein, the training field list includes training fields and training attribute values ​​corresponding to each training field; The generation module 430 is used to adjust the model parameters of the visual encoder and control the other model parameters in the pre-built language visual model to remain unchanged based on each field in the training field list and the preset field list and the attribute values ​​corresponding to the fields, so as to obtain the object detection model.

[0101] Therefore, by constructing data samples and fine-tuning the pre-built language visual model, the resulting target detection model can quickly achieve fine-grained detection of target users and related objects, thereby improving the accuracy of subsequent target detection and enhancing the effectiveness of legacy reminders.

[0102] Optionally, the generation module 430 is specifically used for: calculating multiple field similarities between the training attribute value corresponding to each training field and the label attribute value corresponding to each preset field; calculating a training loss value based on a preset loss function and the multiple field similarities; adjusting the model parameters of the visual encoder and controlling other model parameters in the pre-built language visual model to remain unchanged based on the training loss value and a preset loss threshold, until the training loss value is less than the preset loss threshold, thereby obtaining the object detection model.

[0103] Therefore, by determining the loss value based on the similarity between the preset fields and the fields processed by the model, the model parameters are continuously adjusted to obtain the final object detection model. This allows the object detection model to accurately acquire each field and the attribute information of each field, further improving the accuracy of subsequent object detection.

[0104] The target detection model training device provided in this disclosure can execute the target detection model training method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0105] Figure 5 This is a schematic diagram of a target detection processing device provided in an embodiment of the present disclosure. This device can be implemented by software and / or hardware, and is generally integrated into an electronic device. Figure 5 As shown, the device includes: The second acquisition module 510 is used to acquire a cabin image inside the vehicle cabin; the processing module is used to input the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list; wherein, the target user field list is used to represent target user attribute information and the candidate associated object field list is used to represent candidate associated object attribute information; Analysis module 520 is used to perform analysis based on the target user field list and the candidate associated object field list, and obtain candidate associated objects with a preset association relationship with the target user as target associated objects; The reminder module 530 is used to obtain the target associated object in the vehicle cabin when the target user leaves the vehicle cabin, and trigger a leave reminder; wherein the leave reminder is used to remind that the target associated object has been left in the vehicle cabin.

[0106] Therefore, by processing cabin images through a target detection model, it is possible to accurately identify target objects in the vehicle cabin that have a pre-defined association with the target user, and to provide a reminder when the target user leaves the vehicle cabin but the target object is still in the vehicle cabin, thereby improving the accuracy of target detection and the effectiveness of the reminder.

[0107] Optionally, the target user field list includes target user fields and target user attribute values ​​corresponding to the user fields, and the candidate associated object field list includes candidate associated object fields and candidate associated object attribute values ​​corresponding to the candidate associated object fields. The analysis module 520 is specifically used for: determining target user attribute information based on the target user attribute values ​​corresponding to each target user field in the target user field list; wherein the target user attribute information includes one or more of target user identifier, target user location, target user behavior, and target user image location identifier; determining candidate associated object attribute information based on the candidate associated object attribute values ​​corresponding to each candidate associated object field in the candidate associated object field list; wherein the candidate associated object attribute information includes one or more of candidate associated object identifier, candidate associated object category, candidate associated object image location identifier, candidate associated object color, candidate associated object material, candidate associated object brand, and the relationship between the candidate associated object and the target user; determining the association relationship between the target user and the candidate associated object based on the target user attribute information and the candidate associated object attribute information, and taking the candidate associated object whose association relationship is the preset association relationship as the target associated object.

[0108] Therefore, by using a target detection model to accurately acquire multi-dimensional, fine-grained information within the vehicle cabin, such as user attribute information like target user identifiers, locations, and behaviors, as well as related object attribute information like associated object identifiers, categories, colors, brands, and the relationship between the associated object and the target user, the relationship between the associated object and the target user can be accurately determined. Based on this relationship, target associated objects with a pre-defined association with the target user can be identified, thereby improving the efficiency and effectiveness of target detection.

[0109] Optionally, the reminder module 530 is specifically used to include: acquiring a current cabin image inside the vehicle cabin, and recognizing the current cabin image to obtain a recognition object; when it is determined that the target associated object exists in the recognition object, determining that the target associated object exists in the vehicle cabin, and triggering a leave reminder; wherein the leave reminder is used to remind that the target associated object has been left in the vehicle cabin.

[0110] Therefore, when the target user leaves the vehicle cabin, the system obtains the identification objects present in the current vehicle cabin through the current cabin image, and determines the presence of the target associated object in the vehicle cabin when it judges that the target associated object exists among the identification objects, thus ensuring the accuracy of the identification and the effectiveness of subsequent residual reminders, and meeting the user's usage needs.

[0111] Optionally, the device further includes: an acquisition and determination module, configured to determine target associated object attribute information based on the target associated object field list of the target associated object; wherein the target associated object attribute information includes one or more of the following: target associated object identifier, target associated object category, target associated object image location identifier, target associated object color, target associated object brand, and the relationship between the target associated object and the target user; the module determines a target reminder method based on the target associated object attribute information; and triggers the legacy reminder based on the target reminder method.

[0112] Therefore, when it is determined that there is a target associated object in the vehicle cabin, the corresponding target reminder method is determined based on one or more target associated object attribute information such as target object identifier, category, color, brand, and the relationship between the target associated object and the target user. The legacy reminder is then triggered according to the target reminder method, thereby enabling different reminder methods to be selected for legacy reminders based on different target associated objects, ensuring that legacy reminders can be accurately received by the user, and further improving the flexibility and effectiveness of legacy reminders.

[0113] Optionally, after obtaining the target associated object in the vehicle cabin, the device further includes: a detection module, used to trigger the residual reminder according to a preset first reminder strategy when the door is detected to be closed; or, the detection module is used to trigger the residual reminder according to a preset second reminder strategy when the door is detected to be open.

[0114] Therefore, after obtaining information about the presence of a target-related object in the vehicle's cabin, the system detects the door status and determines the distance between the target user and the vehicle. Based on the vehicle status, different reminder strategies are selected to trigger legacy reminders, ensuring that the target user can effectively receive legacy reminders and guaranteeing their effectiveness.

[0115] Optionally, when the detected door is not closed, the target associated object is an intelligent in-vehicle machine device with motion functions, and the device further includes: a sending module, used to send an action enable command to the target associated object; wherein, the action enable command is used to control the target associated object to activate the motion function and leave the vehicle cabin.

[0116] Therefore, when the detection door is not closed and the target associated object is an intelligent in-vehicle machine with motion function, the target associated object can be directly controlled to start the motion function and leave the vehicle cabin, which enriches the interaction between the vehicle and the intelligent in-vehicle device while improving the convenience of control.

[0117] The target detection processing apparatus provided in this disclosure can execute the target detection processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0118] This disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the target detection model training method and the target detection processing method provided in any embodiment of this disclosure.

[0119] Figure 6 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this disclosure.

[0120] For example, such as Figure 6 As shown, the vehicle includes a memory 601 and a processor 602. The memory 601 stores executable program code 6011, and the processor 602 is used to call and execute the executable program code 6011 to perform a target detection processing method.

[0121] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each module can correspond to a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0122] When each functional module is divided according to its corresponding function, the electronic device may include: a receiving and acquiring module, a calculation module, and an early warning module, etc. It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0123] The electronic device provided in this embodiment is used to execute the above-described target detection processing method, and thus can achieve the same effect as the above-described implementation method.

[0124] When using integrated units, the electronic device may include a processing module and a storage module. The processing module is used to control and manage the operation of the electronic device. The storage module is used to support the execution of program code and data by the electronic device.

[0125] The processing module may be a processor or a controller that can implement or execute various exemplary logic blocks, modules, and circuits as described herein. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module may be a memory.

[0126] This embodiment also provides a computer-readable storage medium (including but not limited to disk storage, CD-ROM, optical storage, etc.) storing computer program code. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement the target detection model training method and target detection processing method provided in the above embodiment.

[0127] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the target detection model training method and target detection processing method provided in the above embodiment.

[0128] The beneficial effects of the above embodiments can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0129] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0130] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. In the description of this disclosure, it should be understood that if the terms "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the position or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure.

[0131] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element. The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A method for training an object detection model, characterized in that, The method comprises: acquiring an image sample and prompt text; wherein the image sample has corresponding target format information text; wherein the target format information text comprises a preset field list, the preset field list comprising preset fields and label attribute values corresponding to each of the preset fields; inputting the image sample and the prompt text into a pre-constructed language visual model, processing the image sample based on the prompt text based on a visual encoder in the pre-constructed language visual model to obtain a training field list; wherein the training field list comprises training fields and training attribute values corresponding to each of the training fields; adjusting model parameters of the visual encoder based on each field in the training field list and the preset field list and the attribute values corresponding to the fields and controlling other model parameters in the pre-constructed language visual model to remain unchanged to obtain the target detection model.

2. The method of claim 1, wherein, The method comprises: calculating a plurality of field similarities of the training attribute values corresponding to each of the training fields and the label attribute values corresponding to each of the preset fields; calculating based on a preset loss function and the plurality of field similarities to obtain a training loss value; adjusting the model parameters of the visual encoder based on the training loss value and a preset loss threshold and controlling other model parameters in the pre-constructed language visual model to remain unchanged until the training loss value is less than the preset loss threshold to obtain the target detection model.

3. A target detection processing method characterized by comprising: The method comprises: acquiring a cabin image in a vehicle cabin and inputting the cabin image into a target detection model to obtain a target user field list and a candidate associated object field list; wherein the target user field list is used to represent target user attribute information and the candidate associated object field list is used to represent candidate associated object attribute information; wherein the target detection model is obtained according to the target detection model training method of claim 1 or 2; analyzing based on the target user field list and the candidate associated object field list to obtain a candidate associated object having a preset association relationship with the target user as a target associated object; in the case that the target user leaves the vehicle cabin, obtaining that the target associated object exists in the vehicle cabin to trigger a left-behind reminder; wherein the left-behind reminder is used to remind that the target associated object is left behind in the vehicle cabin.

4. The method of claim 3, wherein, The target user field list comprises target user fields and target user attribute values corresponding to the user fields, the candidate associated object field list comprises candidate associated object fields and candidate associated object attribute values corresponding to the candidate associated object fields, and the analysis based on the target user field list and the candidate associated object field list to obtain a candidate associated object having a preset association relationship with the target user as a target associated object comprises: determine target user attribute information based on a target user attribute value corresponding to each of the target user fields in the target user field list, wherein the target user attribute information comprises one or more of a target user identifier, a target user location, a target user behavior, and a target user image location identifier; determine candidate associated object attribute information based on a candidate associated object attribute value corresponding to each of the candidate associated object fields in the candidate associated object field list, wherein the candidate associated object attribute information comprises one or more of a candidate associated object identifier, a candidate associated object category, a candidate associated object image location identifier, a candidate associated object color, a candidate associated object material, a candidate associated object brand, and a candidate associated object-target user relationship; determine an association relationship between the target user and the candidate associated object based on the target user attribute information and the candidate associated object attribute information, and take the candidate associated object whose association relationship is the preset association relationship as the target associated object.

5. The method of claim 3, wherein, The obtaining that the target associated object exists in the vehicle cabin comprises: obtaining a current cabin image in the vehicle cabin, and identifying the current cabin image to obtain an identified object; when it is determined that the identified object has the target associated object, it is determined that the target associated object exists in the vehicle cabin.

6. The method of claim 3, wherein, The method further comprises: determining target associated object attribute information based on a target associated object field list of the target associated object, wherein the target associated object attribute information comprises one or more of a target associated object identifier, a target associated object category, a target associated object image location identifier, a target associated object color, a target associated object brand, and a target associated object-target user relationship; determining a target reminding manner according to the target associated object attribute information, and triggering the left-behind reminding based on the target reminding manner.

7. The method of claim 3, wherein, After the target associated object is obtained to exist in the vehicle cabin, the method further comprises: when it is detected that the vehicle door is in a closed state, triggering the left-behind reminding according to a preset first reminding strategy; or, when it is detected that the vehicle door is not in a closed state, triggering the left-behind reminding according to a preset second reminding strategy.

8. The method of claim 3, wherein, When it is detected that the vehicle door is not in a closed state, the target associated object is a smart vehicle-mounted machine device with a motion function, and the method further comprises: sending a motion enabling instruction to the target associated object, wherein the motion enabling instruction is used to control the target associated object to start the motion function and leave the vehicle cabin.

9. A vehicle characterized by comprising: The vehicle comprises: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the method of any one of claims 1-8.

Images