Vehicle theft identification methods, vehicle theft identification devices, electronic equipment and products

Abstract

This application relates to the field of vehicle security monitoring technology, and provides a vehicle theft identification method, a vehicle theft identification device, an electronic device, and a product. The vehicle theft identification method includes: acquiring a multimodal retrieval vector and a scene text vector of the vehicle in the current scene; based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene, retrieving positive sample reference cases and negative sample reference cases from a historical case database, wherein each positive sample reference case refers to a positive sample case similar to the current scene, and each negative sample reference case refers to a negative sample case similar to the current scene; and based on each positive sample reference case and each negative sample reference case, identifying the theft event of the vehicle. This application can solve the problems of existing model-based theft identification methods lacking external evidence, having insufficient generalization, and potentially producing illusions.

Description

Technical Field

[0001] This application belongs to the field of vehicle security monitoring technology, and in particular relates to a vehicle theft identification method, vehicle theft identification device, electronic equipment and products. Background Technology

[0002] With the rapid development of the transportation and logistics industry, the identification and early warning of vehicle theft has become a crucial requirement for fleet management. Traditional identification systems often employ lightweight, edge-based classification models, which, limited by onboard computing power and training data, suffer from insufficient generalization in new modus operandi or complex, ambiguous scenarios, easily leading to false positives or false negatives. For example, the visual features of horseplay inside a vehicle can be confused with those of a fight, or the normal transfer of goods with a robbery, making it difficult to distinguish between them using simple classification models alone.

[0003] Some solutions attempt to incorporate large language models for analysis, hoping to leverage the generalization capabilities of these models. However, two types of risks remain in practical implementation: one risk stems from model illusion, where large models may fill in missing details, leading to unverifiable conclusions; the other risk arises from the lack of external evidence for the judgment, making it difficult to explain why it was theft or not, even if the conclusion is correct. Summary of the Invention

[0004] This application provides a vehicle theft identification method, vehicle theft identification device, electronic device, and product, which can solve the problems of existing model-based theft identification lacking external evidence, having insufficient generalization, and potentially producing illusions.

[0005] In a first aspect, embodiments of this application provide a method for identifying vehicle theft, including: Obtain the vehicle's multimodal retrieval vector and scene text vector in the current scene; Based on the vehicle's multimodal retrieval vector and scene text vector in the current scene, each positive sample reference case and each negative sample reference case are retrieved from the historical case library. Each positive sample reference case refers to a positive sample case similar to the current scene, and each negative sample reference case refers to a negative sample case similar to the current scene. Based on each of the positive sample reference cases and each of the negative sample reference cases, theft events of the vehicle are identified.

[0006] In this embodiment, by obtaining the multimodal retrieval vector and scene text vector of the vehicle in the current scene, and based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene, positive sample reference cases and negative sample reference cases similar to the current scene are retrieved from the historical case library. This provides a traceable external basis for vehicle theft identification. Furthermore, based on the positive sample reference cases and negative sample reference cases, vehicle theft event identification can effectively distinguish behaviors that are easily confused in visual features but are completely different in nature (such as fighting and brawling, passing items and robbery). This enhances the generalization ability of complex scenes and can also tightly anchor the entire vehicle theft identification process to the retrieved specific historical cases (i.e., positive sample reference cases and negative sample reference cases), rather than letting the model generate freely. This effectively suppresses the risk of the model generating illusions due to filling in missing details.

[0007] In some embodiments of the first aspect, retrieving each positive sample reference case and each negative sample reference case from the historical case library based on the vehicle's multimodal retrieval vector and scene text vector in the current scene includes: For each candidate case in the historical case library, the mixed similarity between the current scene and the candidate case is calculated based on the vehicle's multimodal retrieval vector in the current scene, the vehicle's scene text vector in the current scene, the candidate case's multimodal retrieval vector, and the candidate case's scene text vector. Based on the mixed similarity between the current scenario and each of the candidate cases, a preliminary recall candidate set is selected from the historical case library; The preliminary recall candidate set is reordered using the maximum marginal relevance algorithm to obtain each positive sample reference case and each negative sample reference case.

[0008] In some embodiments of the first aspect, the theft event identification of the vehicle based on each of the positive sample reference cases and each of the negative sample reference cases includes: Based on the mixed similarity between the current scene and each of the positive sample reference cases, the average similarity of the positive samples is calculated; Based on the mixed similarity between the current scene and each of the negative sample reference cases, the average similarity of the negative samples is calculated; Calculate the difference between the average similarity of the positive samples and the average similarity of the negative samples, and map the difference to a search propensity score; The theft incident of the vehicle is identified based on the search propensity score.

[0009] In some embodiments of the first aspect, prior to identifying the theft event of the vehicle based on the search propensity score, the method further includes: Based on each of the positive sample reference cases, each of the negative sample reference cases, and the vehicle's multimodal retrieval vector in the current scenario, a comparison prompt is constructed; The comparison prompts are input into the inference model to infer the theft and robbery event, and the inference result for the theft and robbery event is obtained. Based on the search propensity score and the reasoning result, the theft incident of the vehicle is identified.

[0010] In some embodiments of the first aspect, identifying the theft incident of the vehicle based on the retrieval propensity score and the inference result includes: The reasoning confidence and category label are extracted from the reasoning results; Based on the inference confidence and the category label, calculate the inference score for the theft event; Based on the search propensity score and the category label, a consistency penalty coefficient is determined; The search propensity score and the reasoning score are weighted and fused to obtain a fused score; Multiply the fusion score by the consistency penalty coefficient to obtain the risk index; If the risk index is greater than or equal to the risk threshold, then the theft of the vehicle is determined to have occurred.

[0011] In some embodiments of the first aspect, obtaining the multimodal retrieval vector of the vehicle in the current scene includes: Obtain the video feature vector, audio feature vector, and sensor feature vector of the vehicle; The video feature vector, the audio feature vector, and the sensor feature vector are projected onto the same dimension to obtain the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector. The projected video feature vector, the projected audio feature vector, and the projected sensor feature vector are concatenated to obtain a joint feature vector; The joint feature vector is input into the gating calculation unit for linear transformation and normalization to obtain the gating coefficients of the projected video feature vector, the gating coefficients of the projected audio feature vector, and the gating coefficients of the projected sensor feature vector. Based on the gating coefficients of the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector, the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector are weighted and fused to obtain the multimodal retrieval vector of the vehicle in the current scene.

[0012] In some embodiments of the first aspect, obtaining the video feature vector of the vehicle includes: In acquiring the video data of the vehicle The feature vectors and weight coefficients of each sampled frame The weight coefficients of each sampled frame are positively correlated with the corresponding frame index. It is an integer greater than 1; based on The weight coefficients of each of the sampled frames, for The feature vectors of the sampled frames are weighted and fused to obtain the video feature vector.

[0013] Secondly, embodiments of this application provide a vehicle theft identification device, including: The vector acquisition module is used to acquire the vehicle's multimodal retrieval vector and scene text vector in the current scene; The case retrieval module is used to retrieve each positive sample reference case and each negative sample reference case from the historical case library based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene. Each positive sample reference case refers to each positive sample case similar to the current scene, and each negative sample reference case refers to each negative sample case similar to the current scene. The theft identification module is used to identify theft events of the vehicle based on each of the positive sample reference cases and each of the negative sample reference cases.

[0014] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the vehicle theft identification method as described in any of the first aspects above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the vehicle theft identification method as described in any one of the first aspects above.

[0016] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the vehicle theft identification method as described in any one of the first aspects above to be executed.

[0017] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a vehicle theft identification method provided in an embodiment of this application; Figure 2 This is another schematic diagram of the vehicle theft identification method provided in the embodiments of this application; Figure 3 This is another flowchart illustrating the vehicle theft identification method provided in the embodiments of this application; Figure 4 This is another schematic flowchart of the vehicle theft identification method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the vehicle theft identification device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0021] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0025] The vehicle theft identification method provided in this application embodiment can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, desktop computers, servers, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of electronic device.

[0026] To illustrate the technical solution of this application, specific embodiments are described below.

[0027] Please see Figure 1 , Figure 1 The flowchart illustrating a vehicle theft identification method provided in an embodiment of this application is shown as an example and not a limitation. The method includes the following steps: Step 101: Obtain the vehicle's multimodal retrieval vector and scene text vector in the current scene.

[0028] The aforementioned multimodal retrieval vector can refer to a unified vector representation for retrieval generated after multimodal feature extraction and fusion of the vehicle in the current scene.

[0029] The aforementioned scenario text vector can refer to the semantic representation of the current scenario, used for semantic supplementation retrieval. The current scenario can refer to the situation or environment presented by the vehicle in the current time period. Specifically, it can refer to the surveillance footage of the vehicle within the current time period that needs to be identified for theft incidents. This surveillance footage is a multimodal data packet containing multimodal data such as video data, audio data, and sensor data for the current time period. The current time period includes the current moment and at least one historical moment earlier than the current moment. Optionally, the duration of the current time period can be set according to actual needs or empirical values.

[0030] In some embodiments, a text summary of the scene content of the vehicle's video data in the current time period can be used first. This text is then encoded into a semantic representation that can be used for retrieval and comparison. This semantic representation is the scene text vector. The scene text vector is essentially a high-level summary of information such as "what happened, what environment it was in, and what objects and actions were involved" in the vehicle's video data in the current time period. It can compensate for the inadequacy of pure visual features in expressing situations such as occlusion, poor lighting, changes in viewpoint, or high image noise.

[0031] In some embodiments, video footage of the vehicle can be input into a multimodal large model for scene analysis. The multimodal large model automatically generates a natural language description. For example, the multimodal large model might output: "The driver's cabin is dimly lit, and a man dressed in dark clothing is physically threatening the driver, seemingly holding a stick-like object." This natural language description is then input into a text encoding model to convert it into a computer-processable vector form (i.e., a scene text vector). This vector includes key semantic information such as "physical threat," "dimly lit," and "stick-like object."

[0032] Step 102: Based on the vehicle's multimodal retrieval vector and scene text vector in the current scene, retrieve each positive sample reference case and each negative sample reference case from the historical case library.

[0033] In this context, positive sample reference cases refer to positive sample cases similar to the current scenario. Negative sample reference cases refer to negative sample cases similar to the current scenario.

[0034] A historical case library can refer to a case database that includes at least one positive sample case and at least one negative sample case. Both positive and negative sample cases in the historical case library are considered candidate cases; that is, every case in the historical case library can be called a candidate case. The historical case library stores pre-calculated multimodal retrieval vectors and scene text vectors for each candidate case.

[0035] Positive sample cases refer to cases where theft or robbery has been clearly identified (i.e., historical theft or robbery). In the assessment of the current scenario, if a positive sample case is highly similar to a positive sample case, then that positive sample case constitutes positive evidence supporting the determination that aft or robbery has occurred in the current scenario. Negative sample cases refer to cases where theft or robbery has been clearly identified (i.e., historical normal events). In the assessment of the current scenario, if a negative sample case is highly similar to a negative sample case, then that negative sample case constitutes positive evidence supporting the determination that aft or robbery has not occurred in the current scenario.

[0036] In this embodiment, by encoding the current scene into a multimodal retrieval vector and a scene text vector, and based on this, retrieving similar positive and negative sample reference cases from the historical case library, an objective and traceable decision-making basis is laid for the vehicle theft identification process.

[0037] Step 103: Based on each positive sample reference case and each negative sample reference case, identify vehicle theft events.

[0038] Among them, vehicle theft identification can refer to identifying whether a vehicle has been stolen in the current scenario.

[0039] In this embodiment, based on each positive sample reference case and each negative sample reference case, theft and robbery of vehicles are identified. This can transform the identification of vehicle theft and robbery from direct classification to case comparison. As a result, the system no longer relies on a single classification model with high generalization ability that attempts to cover all unknown scenarios. Instead, it places the current scenario to be analyzed in an objective reference system composed of historical facts for comparison.

[0040] In this embodiment, by obtaining the multimodal retrieval vector and scene text vector of the vehicle in the current scene, and based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene, positive sample reference cases and negative sample reference cases similar to the current scene are retrieved from the historical case library. This provides a traceable external basis for vehicle theft identification. Furthermore, based on the positive sample reference cases and negative sample reference cases, vehicle theft event identification can effectively distinguish behaviors that are easily confused in visual features but are completely different in nature (such as fighting and brawling, passing items and robbery). This enhances the generalization ability of complex scenes and can also tightly anchor the entire vehicle theft identification process to the retrieved specific historical cases (i.e., positive sample reference cases and negative sample reference cases), rather than letting the model generate freely. This effectively suppresses the risk of the model generating illusions due to filling in missing details.

[0041] In some embodiments of this application, such as Figure 2 As shown, obtaining the multimodal retrieval vector of the vehicle in the current scene may include steps 201 to 205.

[0042] Step 201: Obtain the vehicle's video feature vector, audio feature vector, and sensor feature vector.

[0043] In this embodiment, video data, audio data, and sensor data of the vehicle in the current time period can be acquired. Video feature vectors can be extracted from the video data, audio feature vectors can be extracted from the audio data, and sensor feature vectors can be extracted from the sensor data.

[0044] In some embodiments, video data can be non-uniformly sampled and time-weighted encoded to obtain the vehicle's video feature vector. Audio Mel-spectral features can be extracted from audio data and determined as audio feature vectors. Sensor data can be directly used as sensor feature vectors.

[0045] In some embodiments, the aforementioned video data may refer to video data inside the vehicle. The aforementioned audio data may refer to audio data inside the vehicle. The aforementioned sensor data includes, but is not limited to, vehicle speed, vibration (e.g., feature vectors extracted from sensor signals such as accelerometers, characterizing the intensity of mechanical impact), and door magnetic state (which may be a binary switch quantity, 0 indicating open and 1 indicating closed), etc.

[0046] Step 202: Project the video feature vector, audio feature vector, and sensor feature vector onto the same dimension to obtain the projected video feature vector, projected audio feature vector, and projected sensor feature vector.

[0047] In this embodiment, projection matrices for the video feature vector, audio feature vector, and sensor feature vector can be preset. Multiplying the projection matrix of the video feature vector by the video feature vector yields the projected video feature vector. Multiplying the projection matrix of the audio feature vector by the audio feature vector yields the projected audio feature vector. Multiplying the projection matrix of the sensor feature vector by the sensor feature vector yields the projected sensor feature vector.

[0048] The formulas for calculating the projected video feature vector, projected audio feature vector, and projected sensor feature vector are as follows:

[0049] in, The projected video feature vector; The projection matrix of the video feature vectors; For video feature vectors; The projected audio feature vector; The projection matrix of the audio feature vectors; For audio feature vectors; The feature vector of the sensor after projection; This is the projection matrix of the sensor feature vectors; This represents the sensor feature vector.

[0050] In this embodiment, by projecting video feature vectors, audio feature vectors, and sensor feature vectors onto the same dimension, heterogeneous features from video, audio, and sensors can be mapped into a unified vector space, making them have the same dimension and comparability. This allows subsequent gating computation units to dynamically and reasonably allocate the weights (i.e., gating coefficients) of each modality based on the content, achieving high-quality feature fusion and ultimately generating more expressive and robust multimodal retrieval vectors.

[0051] Optionally, the dimensions of the projected video feature vector, projected audio feature vector, and projected sensor feature vector can be set according to actual needs or empirical values. For example, the dimensions of the projected video feature vector, projected audio feature vector, and projected sensor feature vector are all 512.

[0052] In this embodiment, a unified multimodal retrieval vector representing the current scene is generated based on the projected video feature vector, projected audio feature vector, and projected sensor feature vector. This vector can jointly characterize a complex scene from multiple complementary perceptual dimensions (visual, auditory, and vehicle physical state), thus generating a scene representation that is more expressive and comprehensive than single-modal features. This comprehensive vector representation provides a richer and more robust comparison foundation for subsequent similarity retrieval in the historical case database. It helps to simultaneously match visual behavior, sound events, and vehicle state during retrieval, thereby improving the relevance and accuracy of retrieval results and providing more reliable underlying feature support for the final identification of theft incidents.

[0053] Step 203: Concatenate the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector to obtain a joint feature vector.

[0054] Step 204: Input the joint feature vector into the gating calculation unit for linear transformation and normalization to obtain the gating coefficients of the projected video feature vector, the gating coefficients of the projected audio feature vector, and the gating coefficients of the projected sensor feature vector.

[0055] The formula for calculating the gating coefficient vector is as follows:

[0056] in, This is the gated weight matrix; This is a gated bias term; Let be the gating coefficient vector, satisfying . , , These are the gating coefficients of the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector, respectively.

[0057] In this embodiment, the gating calculation unit can evaluate and allocate the gating coefficients of each modality in real time based on the specific content of the current input (such as video quality, audio clarity, and sensor signal reliability). When the information of a certain modality is noisy or unreliable, it can automatically reduce its gating coefficient and enhance the influence of other reliable modalities, thereby ensuring that the final generated multimodal retrieval vector can make the most of the effective information and suppress noise interference.

[0058] Step 205: Based on the gating coefficients of the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector, the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector are weighted and fused to obtain the multimodal retrieval vector of the vehicle in the current scene.

[0059] The formula for calculating the multimodal retrieval vector is as follows:

[0060] in, This is a multimodal retrieval vector.

[0061] In this embodiment, the gating coefficient of the multimodal retrieval vector is dynamically adjusted according to the content of the current scene. When the quality of a certain modality is poor, the system will reduce its gating coefficient and rely more on other modalities.

[0062] In this embodiment, by concatenating the projected multimodal feature vectors into a joint feature vector and inputting it into a gating computation unit for linear transformation and normalization, adaptive gating coefficients for each modality can be dynamically generated. Then, based on these coefficients, the corresponding feature vectors are weighted and fused to finally obtain a multimodal retrieval vector. This ensures that the final generated multimodal retrieval vector makes the most of effective information, suppresses noise interference, and generates a unified vector representation that is more representative, stable, and discriminative of complex real-world scenarios. This lays a crucial high-quality feature foundation for subsequent accurate case retrieval and reliable risk assessment.

[0063] In some embodiments of this application, obtaining the video feature vector of the vehicle includes: In acquiring vehicle video data The feature vectors and weight coefficients of each sampled frame The weight coefficients of each sampled frame are positively correlated with the corresponding frame index. It is an integer greater than 1; based on The weight coefficients of each sampled frame, for The feature vectors of each sampled frame are weighted and fused to obtain the video feature vector.

[0064] In this embodiment, the video data can first be non-uniformly sampled to obtain... Each sampled frame, then based on The weight coefficients for each sampled frame are calculated using the frame index of each sampled frame.

[0065] Theft and robbery often become more apparent in the latter part of an event, such as approaching the target before attempting to snatch or control them. Therefore, sampled frames can be assigned linearly increasing weights. The formula for calculating the weight coefficients of sampled frames is as follows:

[0066] in, For the first Frame (i.e., the first) The weight coefficients (dimensionless, with values ​​ranging from (0,1)) of the sampled frames are normalized and linearly increasing. The number of sampling frames (dimensionless); This is the frame index. Optionally, the number of sampling frames can be set according to actual needs or empirical values. For example, the number of sampling frames is 16.

[0067] In this embodiment, by using the above-mentioned formula for calculating the weight coefficients, the later-stage frames in the time sequence can have higher weights, making it easier to capture key action phases.

[0068] The formula for calculating video feature vectors is as follows:

[0069] in, For the first The feature vector of the frame (dimensionless). Optionally, the dimension of the feature vector of the sampled frame can be set according to actual needs or empirical values. For example, the dimension of the feature vector of the sampled frame is 2048.

[0070] In this embodiment, by extracting from video data The feature vectors of each sampled frame are assigned weight coefficients positively correlated with the frame index, and then weighted and fused to obtain a video feature vector. Through a temporal weighting mechanism, the final generated video feature vector can more prominently reflect the key action information that is more discriminative in the later stages of the event, thereby significantly improving the accuracy and robustness of the representation of dynamic behaviors such as theft and robbery. For example, since key behaviors in theft and robbery events (such as snatching and control) are often more obvious and certain in the latter half of the event sequence, assigning higher weights to later frames in the temporal sequence can enable the fused feature vector to better capture and strengthen the visual patterns of these key stages, effectively suppressing the interference of irrelevant or misleading information (such as normal communication and preparatory actions) that may exist in the early stages of the event. This results in a more event-discriminative and temporally-aware video representation for subsequent retrieval and comparison.

[0071] In this embodiment, by introducing a temporal linear weighting and multimodal gating fusion mechanism in the feature extraction stage, it is easier to capture the key actions in the latter half of the theft incident, and the modal contribution can be adaptively adjusted in scenarios such as insufficient light or lack of audio, resulting in a more stable generated feature vector.

[0072] In some embodiments of this application, retrieving each positive sample reference case and each negative sample reference case from the historical case library based on the vehicle's multimodal retrieval vector and scene text vector in the current scene may include steps 301 to 303.

[0073] Step 301: For each candidate case in the historical case library, calculate the mixed similarity between the current scene and the candidate case based on the vehicle's multimodal retrieval vector in the current scene, the vehicle's scene text vector in the current scene, the candidate case's multimodal retrieval vector, and the candidate case's scene text vector.

[0074] To balance visual similarity and semantic relevance, the formula for calculating the mixed similarity between the current scene and a candidate case is as follows:

[0075] in, This is the multimodal retrieval vector for the current scenario; For the multimodal retrieval vector of candidate cases; This is the scene text vector for the current scene; The scene text vectors for candidate cases; The visual and semantic balance coefficient (dimensionless, with a value range of...) Optionally, settings can be configured based on actual needs or experience. .For example, It is 0.7.

[0076] Step 302: Based on the mixed similarity between the current scenario and each candidate case, a preliminary recall candidate set is selected from the historical case library.

[0077] The initial recall candidate set can refer to a batch of candidate cases that are most relevant to the current scenario, selected from the entire historical case database in a preliminary and rapid manner. For example, after sorting the candidate cases in descending order of mixed similarity, the top M candidate cases selected constitute the initial recall candidate set. M is an integer greater than 1, and M is greater than the sum of the number of each positive sample reference case and each negative sample reference case.

[0078] In some embodiments, a hierarchical navigable smallworld (HNSW) index can be used to quickly and accurately retrieve candidate cases that are visually and semantically most similar to the current scene from the historical case library, forming an initial recall candidate set for subsequent maximum marginal relevance (MMR) diversity reordering.

[0079] Step 303: The preliminary recall candidate set is reordered using the maximum marginal relevance algorithm to obtain reference cases for each positive sample and reference cases for each negative sample.

[0080] In this embodiment, by introducing the maximum marginal relevance algorithm to constrain diversity during the retrieval stage, the reference cases cover a wider range of vehicle and environmental conditions (the reference cases cover different vehicles, lighting, and shooting locations), reducing the probability of homogeneous retrieval results and thus providing a more comprehensive reference for comparative reasoning.

[0081] It should be noted that the number of positive sample reference cases and the number of negative sample reference cases can be the same or different, and this application does not impose any limitation on this. For example, the number of positive sample reference cases and the number of negative sample reference cases can both be K, where K is an integer greater than zero. Optionally, K can be set according to actual needs or empirical values.

[0082] In some embodiments, the set of cases can be iteratively selected to match the current scenario. The mixture shows high similarity, while the differences within the set are even greater. Based on this, the next candidate case selected from the initial recall candidate set will be added to the selected set. The calculation formula is as follows:

[0083] in, For the initial recall of candidate sets; The selected set; These are candidate cases to be evaluated. Candidate cases from the selected set; The diversity balance factor (dimensionless, with a value range of...) Optionally, settings can be configured based on actual needs or experience. .For example, It is 0.6.

[0084] As an example, not a limitation, let's take a three-round iteration. First iteration: From the initial candidate set, find candidate case A (e.g., the driver's cab scene) that is most similar to the current scene. First penalty: To enforce diversity, suppress candidate cases highly similar to candidate case A. In subsequent calculations, the scores of these penalized candidate cases will be significantly reduced to prevent them from being selected again. Second iteration: After eliminating distractors highly similar to candidate case A, find the next candidate in the remaining candidate set, finding candidate case B (e.g., the cargo box scene). Second penalty: Penalize candidate cases similar to both candidate case A and candidate case B. Third iteration: Continue the above logic to select candidate case C (e.g., the night scene). After three rounds of iteration, the final selected set includes candidate cases A, B, and C.

[0085] In some embodiments, the number of cases K in the selected set can be preset. In each iteration, the selected set can start from an empty set and add one case at a time until its size reaches the preset number of cases K, at which point the iteration will stop.

[0086] In some embodiments, each positive sample case in the selected set can be directly determined as a positive sample reference case, and each negative sample case in the selected set can be determined as a negative sample reference case.

[0087] In other embodiments, the top K1 positive sample cases with the highest mixed similarity can be selected from each positive sample case in the selected set, and these top K1 positive sample cases are determined as positive sample reference cases. Similarly, the top K2 negative sample cases with the highest mixed similarity can be selected from each negative sample case in the selected set, and these top K2 negative sample cases are determined as negative sample reference cases. K1 and K2 are integers greater than zero and less than K. Optionally, K1 and K2 can be set according to actual needs or empirical values. K1 and K2 can be the same or different.

[0088] In this embodiment, after filtering using the maximum marginal relevance algorithm, positive and negative sample reference cases are obtained. These cases are similar to the current case (i.e., the current scenario) but also cover different environmental conditions. This avoids the selected reference cases being concentrated in a single scenario, thus reducing their reference value and providing a more comprehensive reference for subsequent comparative reasoning. A single scenario can refer to retrieved reference cases that, while "very similar," share almost identical shooting and business environments, resulting in a narrow reference scope and reduced comparative value. For example, if the current video clip (i.e., video data) takes place inside a driver's cab, and only the top few results are selected based on similarity, the search results may all come from footage of "daytime, inside a driver's cab, fixed camera position, same vehicle model / similar perspective" (all resembling physical contact between the driver and passenger, handing over items, or arguments). However, what is more helpful for analysis is to include reference cases that cover situations with different environmental conditions but still have reference value, such as "low-light cab at night", "camera position in cargo box or side door", "external camera position in parking lot / loading point", "different vehicle models, different lens distances and angles, and different degrees of obstruction", etc. In this way, it is easier to extract key differences and common evidence that are valid across scenarios in subsequent comparative reasoning.

[0089] In some embodiments of this application, such as Figure 4 As shown, the theft of a vehicle can be identified based on each positive and negative sample reference case, and may include steps 401 to 404.

[0090] Step 401: Calculate the average similarity of positive samples based on the mixed similarity between the current scene and each positive sample reference case.

[0091] In this embodiment, the average of the mixed similarity between the current scene and each positive sample reference case is calculated, and this average is the average similarity of the positive samples.

[0092] Step 402: Calculate the average similarity of negative samples based on the mixed similarity between the current scene and each negative sample reference case.

[0093] In this embodiment, the average of the mixed similarity between the current scene and each negative sample reference case is calculated, and this average is the negative sample average similarity.

[0094] Step 403: Calculate the difference between the average similarity of positive samples and the average similarity of negative samples, and map the difference to a search propensity score.

[0095] Step 404: Identify vehicle theft incidents based on retrieval propensity scores.

[0096] The search propensity score represents the degree to which the current scenario is likely to be judged as a theft or robbery incident. A higher search propensity score indicates that the current scenario is more likely to be judged as a theft or robbery incident; a lower search propensity score indicates that the current scenario is more likely to be judged as not having experienced a theft or robbery incident.

[0097] The formula for calculating the search propensity score is as follows:

[0098] in, The average similarity of each positive sample (dimensionless, with a value range of ). ); The average similarity of each negative sample (dimensionless, with a value range of ). ); is the scaling factor (dimensionless). This is the Sigmoid function. Optionally, it can be set according to actual needs or empirical values. .For example, It is 5.0.

[0099] In this embodiment, the retrieval propensity score is closer to 1 when the current scene is more similar to the positive sample reference case, and closer to 0 when the current scene is more similar to the negative sample reference case.

[0100] In this embodiment, the average similarity of positive samples and the average similarity of negative samples are calculated, and their difference is taken. This effectively measures whether the current scenario is more similar to historical theft or robbery events or to historical normal events, overcoming the randomness and bias that may arise from comparing a single similarity value or isolated cases. Mapping this difference to a search propensity score further standardizes this propensity into a probabilistic value with clear comparative significance. This allows subsequent identification decisions (whether based directly on threshold judgment or fused with inference results) to be based on stable and traceable quantitative evidence, significantly improving the transparency, auditability, and reliability of the identification process.

[0101] In some embodiments, identifying vehicle theft based on a search propensity score may include: if the search propensity score is greater than or equal to a score threshold, it can be determined that a vehicle theft has occurred; if the search propensity score is less than the score threshold, it can be determined that no vehicle theft has occurred. Optionally, the score threshold can be set according to actual needs or empirical values. For example, the score threshold is 0.6.

[0102] In some embodiments of this application, before identifying vehicle theft events based on retrieval propensity scores, the method further includes: Based on each positive sample reference case, each negative sample reference case, and the vehicle's multimodal retrieval vector in the current scenario, a comparison prompt is constructed; The comparison prompts are input into the inference model to infer the theft and robbery incident, and the inference results for the theft and robbery incident are obtained. Based on search propensity scores, vehicle theft incidents are identified, including: Based on the retrieval propensity score and inference results, the theft of the vehicle is identified.

[0103] In this embodiment, a comparison prompt is constructed based on each positive sample reference case, each negative sample reference case, and the vehicle's multimodal retrieval vector in the current scene. The comparison prompt is then input into the inference model, which can organize the current scene and positive and negative sample reference cases into a structured input, so that the comparison content includes both positive and negative samples and maintains a balance.

[0104] In this embodiment, at least one feature vector from the multimodal retrieval vector of the vehicle in the current scene can be organized into a structured text summary. Reference case summaries are extracted or generated from the metadata associated with each retrieved reference case (pre-stored case descriptions, key feature annotations, key segment locations, etc.). Then, the structured text summaries, the summaries of each positive sample reference case, and the summaries of each negative sample reference case are combined into a comparison prompt. This comparison prompt, along with explicit requirements for the model, is sent to the inference model, allowing the model to output similarities / differences / evidence / conclusions in a specified format. This does not allow the inference model to make arbitrary judgments, but rather to perform comparative reasoning based on given reference cases and base its conclusions on revisable evidence points. Explicit requirements for the model can include specifying that judgments can only be made based on the aforementioned comparison prompts, requiring the model to output which type of reference case the current scene is closer to, what the most critical similarities and differences are, which points are still uncertain and require manual review, and finally, the risk level and reasons.

[0105] As an example, not a limitation, the comparison prompts include text summaries of video feature vectors in the multimodal retrieval vectors, summaries of each positive sample reference case, and summaries of each negative sample reference case. The text summary of the video feature vectors can be a summary of the video data, such as describing the time window in which the video is viewed, the main characters and changes in their actions (e.g., approaching, pulling, controlling, handing over objects), available auxiliary clues (e.g., obvious sounds of argument, changes in door / seal sensor status, abnormal vehicle vibrations), and indicating key time points / keyframe ranges for suggested review. Each positive sample reference case summary clearly states its key behavioral characteristics and the stage in which it occurred (which actions or interactions are the core basis for judging it as theft), and includes the corresponding key segment locations in the case for easy comparison. Each negative sample reference case summary similarly clearly states its key behavioral characteristics and key segment locations to create a comparison of what it resembles / doesn't resemble.

[0106] Optionally, the reasoning model can be set according to actual needs, and this application does not limit the specific type of reasoning model. For example, the reasoning model can be a large language model.

[0107] The reasoning result is the original text output generated by the reasoning model, which contains complete analysis and judgment.

[0108] In this embodiment, the dual verification of retrieval tendency and inference results is performed simultaneously during the decision-making stage, which can avoid misjudgment caused by relying solely on a single module.

[0109] In some embodiments of this application, vehicle theft identification is performed based on retrieval propensity scores and inference results, including: The reasoning confidence and category labels are extracted from the reasoning results; Based on the inference confidence and the category label, calculate the inference score for the theft event; The consistency penalty coefficient is determined based on the retrieval propensity score and category label; The retrieval propensity score and the reasoning score are weighted and fused to obtain the fused score. Multiplying the fusion score by the consistency penalty coefficient yields the risk index; If the risk index is greater than or equal to the risk threshold, then the vehicle has been determined to have been stolen.

[0110] Among these, the category label represents what the conclusion is. The reasoning confidence level represents how confident one is in the conclusion.

[0111] Reasoning score The calculation formula is as follows:

[0112] in, The confidence level for inference (dimensionless, with a range of values ​​of...) ); Category labels (1 indicates theft, 0 indicates normal).

[0113] In this embodiment, the risk index is calculated by weighting and fusing the retrieval propensity score and the reasoning score, and introducing a consistency penalty term. The calculation formula is as follows:

[0114] in, Risk index; For retrieving propensity score (dimensionless, range of values ​​is 10 ... ); The reasoning fraction (dimensionless, with a range of values ​​of...) ); The fusion weights (dimensionless, with a value range of...) ); This is the consistency penalty coefficient. Optionally, it can be set according to actual needs or empirical values. .For example It is 0.4.

[0115] In this embodiment, when and ,or and When the search direction is consistent (i.e., the search tendency and the inference result have the same direction of judgment on the theft event, such as both identifying the theft event as having occurred or both identifying the theft event as not having occurred), let When the two directions are inconsistent (i.e., the retrieval tendency and the inference result have different directions in judging the theft event, for example, one identifies the theft event as having occurred, and the other identifies it as not having occurred), let ,in This is the penalty coefficient. Optionally, it can be set according to actual needs or empirical values. .For example, It is 0.5.

[0116] Optionally, a risk threshold can be set based on actual needs or experience. For example, the risk threshold could be 0.6.

[0117] In this embodiment, the consistent fusion of retrieval propensity score and inference score during the decision-making stage ensures that the system only determines a theft incident has occurred after dual confirmation from both retrieval and inference, and outputs a high-risk alarm. This reduces the risk of false alarms from a single module and improves the verifiability of the judgment. Actively reducing the weight when the two directions are inconsistent makes the system less susceptible to retrieval noise or inference illusions, thus improving the reliability of the alarm.

[0118] In some embodiments, when determining whether a vehicle has been stolen, a corresponding risk level is also determined. For example, the risk level corresponding to a vehicle being stolen is high risk, and the risk level corresponding to a vehicle not being stolen is low risk.

[0119] In some embodiments, when determining whether a vehicle has been stolen, at least one of the multimodal data such as video data, audio data, and sensor data of the current scene can be compared and explained together with the retrieved positive sample reference cases and negative sample reference cases, clearly stating "which parts are similar, where the key differences are, and which time points / key frames are recommended to be reviewed".

[0120] The video data of the current scenario shares similarities with positive sample reference cases (e.g., forcibly taking personal belongings from a driver's cab) in that the approach is sudden, there is obvious pulling / resistance from both parties, and the controlling actions are continuous. The difference between the current scenario and negative sample reference cases (e.g., normal delivery of items / playful fighting) is that the actions of both parties in the negative sample reference cases are smoother and more cooperative, without continuous control or escalating resistance. Based on these comparisons, the system marks the current scenario as high-risk and provides the corresponding key time period and keyframe location in the conclusion, facilitating quick review and verification by operations personnel.

[0121] This application constructs a case retrieval and comparative reasoning-based judgment architecture for vehicle theft incidents. It generates multimodal retrieval vectors through time-series weighted coding and multimodal gating fusion, and selects positive and negative sample reference cases with reference value through the diversity retrieval of the maximum marginal relevance algorithm. Then, it combines the reasoning results with the retrieval tendency for consistency fusion, and outputs the final risk index and verifiable evidence, making the judgment based on evidence and the conclusion easy to audit.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0123] Corresponding to the vehicle theft identification method described in the above embodiments, Figure 5 A schematic diagram of the vehicle theft identification device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0124] Reference Figure 5 The device includes: The vector acquisition module 501 is used to acquire the vehicle's multimodal retrieval vector and scene text vector in the current scene; The case retrieval module 502 is used to retrieve each positive sample reference case and each negative sample reference case from the historical case library based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene. Each positive sample reference case refers to each positive sample case similar to the current scene, and each negative sample reference case refers to each negative sample case similar to the current scene. Theft identification module 503 is used to identify theft events of the vehicle based on each of the positive sample reference cases and each of the negative sample reference cases.

[0125] In some embodiments, the case retrieval module 502 includes: The first calculation unit is used to calculate the mixed similarity between the current scene and the candidate case for each candidate case in the historical case library, based on the multimodal retrieval vector of the vehicle in the current scene, the scene text vector of the vehicle in the current scene, the multimodal retrieval vector of the candidate case, and the scene text vector of the candidate case. The case screening unit is used to screen a preliminary recall candidate set from the historical case library based on the mixed similarity between the current scene and each of the candidate cases; The reference determination unit is used to reorder the preliminary recall candidate set using the maximum marginal relevance algorithm to obtain each positive sample reference case and each negative sample reference case.

[0126] In some embodiments, the theft identification module 503 includes: The second calculation unit is used to calculate the average similarity of positive samples based on the mixed similarity between the current scene and each of the positive sample reference cases; The third calculation unit is used to calculate the average similarity of negative samples based on the mixed similarity between the current scene and each of the negative sample reference cases; The difference mapping unit is used to calculate the difference between the average similarity of the positive samples and the average similarity of the negative samples, and to map the difference into a search propensity score; Theft identification unit is used to identify theft events of the vehicle based on the search propensity score.

[0127] In some embodiments, the theft identification module 503 further includes: The prompt construction unit is used to construct a comparison prompt based on each of the positive sample reference cases, each of the negative sample reference cases, and the multimodal retrieval vector of the vehicle in the current scene; The reasoning unit is used to input the comparison prompts into the reasoning model to reason about the theft and robbery event, and to obtain the reasoning result for the theft and robbery event. The theft detection unit is specifically used for: Based on the search propensity score and the reasoning result, the theft incident of the vehicle is identified.

[0128] In some embodiments, the theft identification unit is specifically used for: The reasoning confidence and category label are extracted from the reasoning results; Based on the inference confidence and the category label, calculate the inference score for the theft event; Based on the search propensity score and the category label, a consistency penalty coefficient is determined; The search propensity score and the reasoning score are weighted and fused to obtain a fused score; Multiply the fusion score by the consistency penalty coefficient to obtain the risk index; If the risk index is greater than or equal to the risk threshold, then the theft of the vehicle is determined to have occurred.

[0129] In some embodiments, the vector acquisition module 501 includes: The feature acquisition unit is used to acquire the video feature vector, audio feature vector, and sensor feature vector of the vehicle. The feature projection unit is used to project the video feature vector, the audio feature vector and the sensor feature vector onto the same dimension to obtain the projected video feature vector, the projected audio feature vector and the projected sensor feature vector. The feature stitching unit is used to stitch together the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector to obtain a joint feature vector; The feature processing unit is used to input the joint feature vector into the gating calculation unit for linear transformation and normalization to obtain the gating coefficients of the projected video feature vector, the gating coefficients of the projected audio feature vector, and the gating coefficients of the projected sensor feature vector. The weighted fusion unit is used to perform weighted fusion on the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector based on the gating coefficients of the projected video feature vector, the gating coefficients of the projected audio feature vector, and the gating coefficients of the projected sensor feature vector, to obtain the multimodal retrieval vector of the vehicle in the current scene.

[0130] In some embodiments, the feature acquisition unit is specifically used for: In acquiring the video data of the vehicle The feature vectors and weight coefficients of each sampled frame The weight coefficients of each sampled frame are positively correlated with the corresponding frame index. It is an integer greater than 1; based on The weight coefficients of each of the sampled frames, for The feature vectors of the sampled frames are weighted and fused to obtain the video feature vector.

[0131] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0132] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown in the diagram), memory 61, and computer program 62 stored in said memory 61 and executable on said at least one processor 60, wherein said processor 60 executes said computer program 62 to implement the steps in any of the above method embodiments.

[0133] The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0134] The processor 60 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0135] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. In other embodiments, the memory 61 may be an external storage device of the electronic device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 6. Furthermore, the memory 61 may include both internal and external storage units of the electronic device 6. The memory 61 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying vehicle theft, characterized in that, include: Obtain the vehicle's multimodal retrieval vector and scene text vector in the current scene; Based on the vehicle's multimodal retrieval vector and scene text vector in the current scene, each positive sample reference case and each negative sample reference case are retrieved from the historical case library. Each positive sample reference case refers to a positive sample case similar to the current scene, and each negative sample reference case refers to a negative sample case similar to the current scene. Based on each of the positive sample reference cases and each of the negative sample reference cases, theft events of the vehicle are identified.

2. The vehicle theft identification method according to claim 1, characterized in that, The step of retrieving each positive sample reference case and each negative sample reference case from the historical case database based on the vehicle's multimodal retrieval vector and scene text vector in the current scenario includes: For each candidate case in the historical case library, the mixed similarity between the current scene and the candidate case is calculated based on the vehicle's multimodal retrieval vector in the current scene, the vehicle's scene text vector in the current scene, the candidate case's multimodal retrieval vector, and the candidate case's scene text vector. Based on the mixed similarity between the current scenario and each of the candidate cases, a preliminary recall candidate set is selected from the historical case library; The preliminary recall candidate set is reordered using the maximum marginal relevance algorithm to obtain each positive sample reference case and each negative sample reference case.

3. The vehicle theft identification method according to claim 2, characterized in that, The process of identifying vehicle theft events based on each of the positive and negative sample reference cases includes: Based on the mixed similarity between the current scene and each of the positive sample reference cases, the average similarity of the positive samples is calculated; Based on the mixed similarity between the current scene and each of the negative sample reference cases, the average similarity of the negative samples is calculated; Calculate the difference between the average similarity of the positive samples and the average similarity of the negative samples, and map the difference to a search propensity score; The theft incident of the vehicle is identified based on the search propensity score.

4. The vehicle theft identification method according to claim 3, characterized in that, Before identifying the theft event of the vehicle based on the search propensity score, the method further includes: Based on each of the positive sample reference cases, each of the negative sample reference cases, and the vehicle's multimodal retrieval vector in the current scenario, a comparison prompt is constructed; The comparison prompts are input into the inference model to infer the theft and robbery event, and the inference result for the theft and robbery event is obtained. The process of identifying theft incidents of the vehicle based on the search propensity score includes: Based on the search propensity score and the reasoning result, the theft incident of the vehicle is identified.

5. The vehicle theft identification method according to claim 4, characterized in that, The process of identifying theft incidents of the vehicle based on the search propensity score and the inference result includes: The reasoning confidence and category label are extracted from the reasoning results; Based on the inference confidence and the category label, calculate the inference score for the theft event; Based on the search propensity score and the category label, a consistency penalty coefficient is determined; The search propensity score and the reasoning score are weighted and fused to obtain a fused score; Multiply the fusion score by the consistency penalty coefficient to obtain the risk index; If the risk index is greater than or equal to the risk threshold, then the theft of the vehicle is determined to have occurred.

6. The vehicle theft identification method according to any one of claims 1 to 3, characterized in that, The process of obtaining the vehicle's multimodal retrieval vector in the current scene includes: Obtain the video feature vector, audio feature vector, and sensor feature vector of the vehicle; The video feature vector, the audio feature vector, and the sensor feature vector are projected onto the same dimension to obtain the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector. The projected video feature vector, the projected audio feature vector, and the projected sensor feature vector are concatenated to obtain a joint feature vector; The joint feature vector is input into the gating calculation unit for linear transformation and normalization to obtain the gating coefficients of the projected video feature vector, the gating coefficients of the projected audio feature vector, and the gating coefficients of the projected sensor feature vector. Based on the gating coefficients of the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector, the projected video feature vector, the projected audio feature vector, and the projected sensor feature vector are weighted and fused to obtain the multimodal retrieval vector of the vehicle in the current scene.

7. The vehicle theft identification method according to claim 6, characterized in that, The step of obtaining the video feature vector of the vehicle includes: In acquiring the video data of the vehicle Feature vectors and weight coefficients of each sampled frame The weight coefficients of each sampled frame are positively correlated with the corresponding frame index. It is an integer greater than 1; based on The weight coefficients of each of the sampled frames, for The feature vectors of the sampled frames are weighted and fused to obtain the video feature vector.

8. A vehicle theft identification device, characterized in that, include: The vector acquisition module is used to acquire the vehicle's multimodal retrieval vector and scene text vector in the current scene; The case retrieval module is used to retrieve each positive sample reference case and each negative sample reference case from the historical case library based on the multimodal retrieval vector and scene text vector of the vehicle in the current scene. Each positive sample reference case refers to each positive sample case similar to the current scene, and each negative sample reference case refers to each negative sample case similar to the current scene. The theft identification module is used to identify theft events of the vehicle based on each of the positive sample reference cases and each of the negative sample reference cases.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the vehicle theft identification method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, It includes a computer program, which, when run, causes the vehicle theft identification method as described in any one of claims 1 to 7 to be executed.

Images