Mirror direction visual question answering method based on subject center coordinate positioning

Abstract

This invention discloses a mirror-oriented visual question answering method based on subject-center coordinate localization. The method first parses the image, question text q, and candidate option set O, and detects the reference frame category r∈{subject reference frame, camera reference frame, ambiguous reference frame}. When a sample is determined to be in the subject reference frame or an ambiguous reference frame processed according to the subject reference frame, type routing, canonical semantic binding, dedicated solution, deterministic option projection, and consistency gating are performed sequentially. When a sample is determined to be in the camera reference frame, a compatibility-safe path is adopted, selecting the original visual language model to answer to avoid negative transfer from the subject-center path. This method improves the accuracy and consistency of option arrangement in mirror-oriented visual question answering by processing trajectory records of reference frame, question type, canonical semantics, projection, and routing reasons, while reducing reference frame confusion errors and maintaining compatibility across reference frame scenarios.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and multimodal machine learning technology, specifically relating to the field of spatial orientation reasoning technology of visual language models, and more specifically, relating to a mirror orientation visual question answering method based on subject center coordinate positioning. Background Technology

[0002] With the development of Multimodal Large Language Models (MLLM), Visual Question Answering (VQA) has been widely applied in scenarios such as image and text understanding, embodied intelligence, assisted driving, and human-computer interaction. Among these, mirror orientation question answering is a fine-grained spatial reasoning task that requires the model to determine the left-right or front-back orientation of body parts or objects in an image, such as determining whether "the person in the image is raising their left or right hand." This type of task has significant application value in areas such as robot grasping, rehabilitation training assessment, motion posture analysis, and human-computer collaboration.

[0003] The core challenge of mirror-oriented question answering lies in determining the reference frame. Within the same image, there exist two mirror-image coordinate systems: the observer's left and right (image left and right) and the subject's own left and right. When the subject faces the camera, its left side appears precisely on the right side of the image. However, existing multimodal large language models often fail to establish an explicit subject-centered coordinate system when answering questions. Instead, they rely on heuristic judgments of the image's left and right or on prior knowledge of the position of the option letters (A / B / C / D). The inventors' diagnostic experiment on a mirror-oriented question answering benchmark encompassing human activities and animal species revealed two prominent problems with existing open-source multimodal models in this type of task: firstly, reference frame confusion (RFC), where the model, while locating the correct subject or part, confuses the observer's left and right, the image's left and right, and the subject's own left and right; secondly, option position bias, where the model's answer heavily depends on the position of the options. Even after swapping the two options for the same question, the model tends to output letters in the same position, resulting in contradictory answers.

[0004] To address the aforementioned issues, existing technologies suffer from the following shortcomings. First, while prompt-based methods can incorporate guiding phrases such as "distinguish between the left and right sides of the image and the left and right sides of the subject," this often comes at the cost of accuracy and fails to fundamentally eliminate the interference of prior information about option letters on visual judgment. Second, methods based on general fine-tuning or reinforcement learning, if directly targeting the final option letters for optimization, are prone to transferring statistical priors of certain option letters into new answer biases. Third, existing methods generally lack explicit modeling of the reference frame, using the same judgment logic indiscriminately for subject-based and camera / observer-based tasks; when models trained for the subject-based reference frame are directly transferred to external tasks asking questions from the camera's perspective, negative transfer can easily occur. Therefore, there is an urgent need for a mirror-oriented visual question-answering method that can explicitly model the reference frame, decouple visual semantic judgment from option letters, and avoid negative transfer in cross-reference frame scenarios. Summary of the Invention

[0005] In view of this, the present invention provides a mirror-oriented visual question answering method based on subject-center coordinate grounding. Its core concept is to model mirror-oriented question answering as a subject-center coordinate grounding (SCCG) problem. Specifically, it first determines the reference frame, then predicts the canonical semantics independent of the option letters, and finally deterministically projects the canonical semantics onto the current option. Furthermore, it avoids negative migration through compatibility routing in cross-reference frame scenarios. The electronic devices, storage media, or processing modules mentioned herein are all implementation vehicles for running this method and are not intended as independent system solutions limiting the scope of protection of this invention.

[0006] First, this invention proposes a reference frame-driven processing method. By performing reference frame detection on the problem text, the sample is divided into a subject reference frame, a camera reference frame, and an ambiguous reference frame. Based on these, the sample is routed to either a subject center coordinate positioning path or a compatibility and safety path, thereby preventing the misapplication of the judgment logic for the subject reference frame to camera reference frame samples from the source.

[0007] Second, this invention proposes a method for standard semantic binding and deterministic option projection. The constraint model first outputs standard semantic labels taken from a fixed set of labels that are unrelated to the option letters. Then, through text normalization and exact matching, the standard semantics are deterministically mapped to the current option letters. When there are no matching options, the projection is judged as a failure instead of forced guessing, thus structurally eliminating the interference of the prior of the option letters on the visual semantic judgment.

[0008] Third, this invention proposes a consistency gating and non-degenerate compatibility routing method. By performing majority voting consistency verification on the canonical semantics of samples and their option permutations, it backs down to a robust branch when there is inconsistency. Furthermore, it adopts a non-degenerate safe route for camera reference frame samples, selecting the original model to answer, ensuring that the external camera reference frame task does not experience negative migration due to subject center adaptation.

[0009] Fourth, this invention further provides an optional training method. First, supervised fine-tuning enables the model to learn the output format and basic orientation judgment ability of "standardizing semantics first and then selecting letters". Then, through reinforcement learning with group relative policy optimization, the format, standard semantic correctness, option arrangement consistency and non-degeneration are used as multi-dimensional rewards for joint optimization, which further improves the robustness of the reference frame.

[0010] The technical approach of this invention is as follows: First, the input samples undergo structured parsing and reference frame detection; second, the main reference frame samples are sequentially processed using type routing, canonical semantic binding, dedicated solution, deterministic option projection, and consistency gating; the camera reference frame samples are processed using original model answering and non-degenerate safe routing; finally, the final answer with a traceable path is output. The specific steps are as follows:

[0011] S1: Obtain the input image, question text q, and candidate options. Perform structured parsing on the samples to obtain the target subject, target body parts or objects, spatial relationships, and option semantics. Perform reference frame detection on the question text to obtain the subject reference frame, camera reference frame, or ambiguous reference frame.

[0012] S2: For the main reference frame sample and the ambiguous reference frame sample processed by default according to the main reference frame, perform the main center coordinate positioning path, and perform type routing, normalized semantic binding, special solution, deterministic option projection and consistency gating in sequence to obtain the prediction result of the sample.

[0013] S3: Perform a compatibility-safe path on the camera reference frame samples, use the original visual language model to respond, and select the prediction results through non-degenerate safe routing to avoid negative transfer.

[0014] S4: Output the final option letter and record the processing path and routing reason of the sample as a traceable basis.

[0015] Compared with existing technologies, the advantages of this invention are as follows: by decoupling explicit reference frame modeling and canonical semantics-options, it simultaneously improves accuracy and option arrangement consistency on the mirror-direction question-answering benchmark, and reduces reference frame confusion errors; by using non-degenerate compatibility routing, it avoids negative migration of the subject center adapter in tasks outside the camera reference frame; the entire process is traceable, making it easy to locate the source of errors, and taking into account inference accuracy, robustness and interpretability. Attached Figure Description

[0016] Figure 1 is a flowchart illustrating the steps of the mirror-oriented visual question-answering method based on the subject center coordinate positioning of the present invention.

[0017] Figure 2 is a schematic diagram of the overall framework of the mirror-direction visual question answering method based on the subject center coordinate positioning of the present invention; Detailed Implementation

[0018] To enable those skilled in the art to better understand the present invention and to make the above-mentioned objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of the mirror-oriented visual question-answering method based on subject center coordinate positioning according to the present invention. The method includes the following steps:

[0020] S1. Perform structured parsing on the input sample x=(I,q,O), extracting the target subject, target body parts or target objects, spatial relationships, and candidate option semantics, and obtain the reference frame category r through the reference frame detection function fref(q); S2. When r=Fs or r=Fu and is routed to the subject reference frame, execute the subject center coordinate positioning path, obtain the question type t through the type routing function ftype(x), and generate a standardized semantic label ŷ unrelated to the option letters in the corresponding fixed label set Yt; S3. Through deterministic voting... Shadow π(ŷ,O) maps the canonical semantic label ŷ to the option letter â in the current candidate options, and verifies the canonical semantic stability under option exchange or rotation transformation through the consistency gating G; S4, when r=Fc, execute the compatibility safety path, use the original visual language model to obtain the answer araw, and select araw as the prediction result through non-degenerate routing; S5, output the final option letter a* determined by S2 to S4, and record the processing trajectory T=(r,t,ŷ,π,G,route) as a traceable basis.

[0021] Please see Figure 2 , Figure 2This is a schematic diagram of the overall framework of the mirror-oriented visual question answering method based on subject center coordinate positioning according to the present invention. The overall framework includes an input module, a parsing module, a reference frame detection module, a subject reference frame SCCG path, a compatibility and safety path, and a final answer module. The input module receives images, question text, and candidate options; the parsing module extracts body parts, candidate options, and semantic information; the reference frame detection module identifies the subject, camera, or ambiguous reference frame; when a sample enters the subject reference frame SCCG path, stable options are obtained sequentially through type routing, semantic binding, dedicated solving, option projection, and consistency gating; when a sample enters the compatibility and safety path, non-degenerate output on the camera reference frame task is maintained through the original visual language model and safe routing; the final answer module outputs the option letters and retains the answer trajectory and routing information.

[0022] Formal definition

[0023] For ease of description, let the input sample be x=(I,q,O), where I is the image, q is the question text, and O={(a j ,o j )} j=1 m For the set of candidate options; a j The letter o represents the option. j Let represent the option text, and m represent the number of options. The reference frame detection function, type routing function, canonical semantics generation function, and option projection function are respectively represented as follows:

[0024] r = f ref (q), r ∈ {F s , F c , F u} (1)

[0025] t = f type (x), t ∈ {H2, A4, T}, ŷ = g θ (I, q, O, t), ŷ ∈ Y t (2)

[0026] â = π(ŷ, O) = a j , st n(o j ) = n(ŷ) (3)

[0027] Where F s F represents the main reference frame. c F represents the camera reference frame. u Indicates ambiguous reference frames; H2 represents a two-choice question about the human body, A4 represents a four-choice question about animals, and T represents a text-based judgment question; Y tLet n be the set of fixed-standard semantic tags corresponding to question type t, and n(·) be the text normalization function.

[0028] S1, Analysis and Reference Frame Detection

[0029] S1 first performs structured parsing on the input samples. Specifically, it separates the image, the question text q, and the candidate option set O from the samples, and performs semantic parsing on the question text to extract the target subject, target body part or target object, spatial relationship or action, and the directional semantics corresponding to each candidate option. The parsing results are used for subsequent type routing and canonical semantic binding.

[0030] Then, reference frame detection is performed on the question text q to determine in which coordinate system the model is required to answer the question. Specifically, two language pattern sets are pre-configured: a camera reference frame pattern set P. c With the main reference frame model set P s P c Includes language patterns representing camera perspective, observer's perspective, viewer's perspective, or image perspective, such as "from the camera's perspective," "viewer's perspective," "observer's perspective," etc.; P s It includes language patterns that represent the perspective of the subject itself, such as "subject's perspective", "person's perspective", "animal's perspective", "its own left / right", etc.

[0031] The reference frame determination rule is: compare the problem text q with P respectively. c and P s Perform a match; when q hits P c When q hits P, it is determined to be the camera reference frame; s When q is determined to be the main reference frame; when q does not hit P c P was also not hit. s If the reference frame is deemed ambiguous, it is treated as the primary reference frame by default, while retaining a fallback risk flag. In one embodiment, the language pattern is implemented using regular expressions and matched in a case-insensitive manner; in another embodiment, reference frame detection can also be implemented using a lightweight classifier.

[0032] S2, Main body center coordinate positioning path

[0033] S2 performs five sub-steps sequentially on the main reference frame samples: type routing, canonical semantic binding, dedicated solution, deterministic option projection, and consistency gating.

[0034] S21. Type Routing. Based on the subject category and the number of candidate options, the sample is routed to one of the following types: Human (two-choice), Animal (four-choice), or Text Judgment. The fixed label set for the Human (two-choice) type is Y. H2 ={left, right}; The fixed label set for the animal four-choice type is Y. A4 ={left front, right front, left back, right back}; The fixed set of tags for text judgment type is Y. T ={Yes, No}. Different types correspond to different dedicated solvers and label sets, thereby achieving fine-grained division of capabilities.

[0035] S22. Standardized Semantic Binding. Under a defined type, before providing the option letter, the constraint model first outputs a standardized semantic label taken from a fixed label set for that type in a preset format. In one embodiment, the preset format is "Semantic:<standardized semantic label>". The values ​​of the standardized semantic labels are limited to the fixed label set and do not depend on the prior letters or their order of the candidate options. By verifying visual semantics first and then binding the option letters, this invention structurally decouples visual judgment from option position.

[0036] S23. Dedicated Solver. A dedicated solver corresponding to the question type is invoked to solve the sample and output standardized semantic labels. In one embodiment, the dedicated solver includes a conservative human two-choice adapter for the human two-choice type and a semantic verifier for the animal four-choice type. The adapter and semantic verifier are obtained by efficiently fine-tuning the parameters of the basic visual language model on training samples with standardized semantic annotations; for types without a dedicated adapter, the original visual language model can answer according to the standardized semantic format.

[0037] S24. Deterministic Option Projection. The canonical semantic label ŷ is deterministically mapped to the candidate option letters of the current sample. Specifically, first, a normalization function n(·) is performed on each candidate option text. Normalization includes removing option letter prefixes, case normalization, removing non-semantic characters, and mapping synonymous locative words. Then, the option that perfectly matches the canonical semantic label is found among the normalized candidate options, and the letter corresponding to that option is used as the projection result. If no matching option exists, the projection is considered a failure, and instead of forced guessing, a fallback strategy is implemented.

[0038] S25. Consistency Gating. To further verify the robustness of option permutations, variants of option permutations are constructed for the same sample: an A / B exchange variant is constructed for two-choice samples, and a candidate option rotation variant is constructed for four-choice samples. Canonical semantic binding and deterministic option projection are performed on the samples and their variants respectively, resulting in a set of canonical semantic labels {ŷ (k)} k=1 KIf a unique mode exists and its votes are not less than the threshold τ, the projected letter corresponding to the mode label is adopted; otherwise, the original visual language model's answer or the preset robust branch is returned.

[0039] G = 1[count(mode({ŷ (k)})) ≥ τ and mode is unique] (4)

[0040] S3, Compatibility and Security Path

[0041] S3 implements a compatibility-safe path for camera reference frame samples. Specifically, for each camera reference frame sample, the original visual language model is used to directly answer the image, question, and options, resulting in the original answer a. raw The non-degradable secure router makes routing decisions for samples: when r=F c When, choose a raw The result of the prediction for this sample, without using the subject's center coordinates to locate the path, is a. sccg .

[0042] The reason for this design is that the subject center coordinate positioning path and its dedicated adapter are trained with respect to the subject reference frame. If they are directly used for samples posed from the camera's perspective, negative transfer may occur. Through non-degenerate secure routing, this invention ensures that the accuracy on camera reference frame samples is no less than that of the original visual language model; the router also records the route source and reason for each sample for auditing and traceability.

[0043] a * = R safe (r, a raw , a sccg ) = a raw if r = F c (5)

[0044] S4. Final Answer Output

[0045] S4 outputs the final option letter based on the result selected by S2 or S3, along with the processing path, including the reference system category, question type, standard semantic label, projection result or backtracking reason, and routing source. This processing path serves as a traceable basis, facilitating error attribution and interpretability analysis of the model's responses.

[0046] S5, Optional Training Methods

[0047] To further enhance the orientation determination and reference frame robustness of the dedicated solver, this invention provides an optional two-stage training method for training the dedicated solver described in S23. This training method does not limit the specific model structure, but rather constrains the consistency of output format, semantic labels, and option arrangement.

[0048] S51. The first stage employs Supervised Fine-tuning (SFT). Based on training samples with canonical semantic annotations, the basic visual language model is subjected to supervised fine-tuning. Question type identifiers and canonical semantic identifiers are introduced into the training samples as control signals, enabling the model to learn the output format of "first outputting the canonical semantic label, then outputting the option letter." In one embodiment, the SFT objective function is:

[0049] L SFT (θ) = -Σ i [log p θ (y i | I i , q i O i , t i ) + λ log p θ (a i | I i , q i O i , t i ,y i (6)

[0050] Where y i Let a be the canonical semantic label of the i-th training sample. i Let λ represent the corresponding option letter, and λ be the weight of the option letter item. This objective aims to make the model prioritize learning canonical semantics rather than directly memorizing the position of the option letters.

[0051] S52. The second stage employs Group Relative Policy Optimization (GRPO) reinforcement learning. For the same input, a set of candidate outputs is sampled from the old policy, and a total reward is constructed, weighted by format reward, semantic correctness reward, option permutation consistency reward, and non-degeneracy reward.

[0052] R i = w f R i fmt + w s R i sem + w c R i cons+ w d R i nd (7)

[0053] Within-group normalization of candidate output rewards for the same input yields the advantage function:

[0054] A i,k = (R i,k - μ i ) / (σ i + δ) (8)

[0055] During policy optimization, the objective function of the group relative policy optimization with shearing is maximized, and a KL divergence regularization term is added to constrain the deviation between the current policy and the reference policy.

[0056] J GRPO (θ) = E[min(ρ i,k A i,k clip(ρ) i,k , 1-ε, 1+ε)A i,k ) - βD KL (π θ || π ref (9)

[0057] Where ρ i,k =π θ (o i,k |x i ) / π old (o i,k |x i ), where ε is the shearing threshold and β is the KL regularization weight. In one embodiment, the training data is proportionally mixed with human two-choice samples, animal four-choice sample rotation samples, animal two-choice samples, and negative samples from the camera reference frame. Strong constraints of option arrangement consistency and non-degeneracy are used as gating mechanisms. A candidate model is only allowed to replace the existing solver if it improves the target metrics without compromising the semantic consistency of the canonical model and the accuracy of each correct option category.

[0058] Examples and Effect Verification

[0059] In one specific embodiment, a mirror-oriented question-and-answer benchmark containing 5549 manually annotated images covering 28 human activities and 43 animal species was used as the evaluation set. The samples were divided into human two-choice, animal two-choice, and animal four-choice categories, and consistency evaluation was constructed using option swapping or round-robin. An open-source multimodal model was used as the original visual language model, with its conservative adapter for human two-choice and semantic validator for animal four-choice obtained through efficient parameter fine-tuning. Experimental results are shown in Table 1.

[0060] Table 1. Experimental results of the method of the present invention on the mirror-oriented visual question answering task.

[0061] Verification project Baseline or control Method / routing result of the application Effect description Complete test set Acc 0.5772 0.6269 +0.0497, accuracy improved by about 4.97 percentage points Complete test set Final F1 0.5963 0.6334 +0.0371, comprehensive F1 improved A / B exchange specification semantic consistency 0.3697 0.7007 +0.3310, option arrangement robustness improved A / B bias gap 0.5612 0.2300 Bias gap reduced by about 0.3312 Number of RFC errors 120 102 Reference system confusion error reduced by 15.00% Number of BPM errors 84 55 Body part misjudgment error reduced by 34.52% Number of OOI errors 36 20 Shielding / directional interference error reduced by 44.44% EgoOrient direct adaptation 0.7600 0.7400 Direct migration decreased by 0.0200, indicating the need for compatible routing EgoOrient compatible routing 0.7600 0.7600 No degradation, avoiding negative migration of camera reference system

[0062] As shown in Table 1, the final solution of this invention improves the accuracy on the complete test set from 0.5772 to 0.6269 compared to the original model, and improves the canonical semantic consistency under option A / B exchange from 0.3697 to 0.7007. Statistically, errors of RFC, BPM, and OOI types all show a decrease. External camera reference frame verification shows that directly using the subject reference frame adapter for camera viewpoint samples results in a decrease in accuracy, while the accuracy remains unchanged after adopting the compatibility-safe routing of this invention, verifying the effectiveness of the non-degradable path. The above results are used to illustrate the technical effects of the method of this invention and do not limit the base model, training data scale, or specific implementation framework.

[0063] Those skilled in the art should understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For example, the fixed label set, language pattern set, specific form of the dedicated solver, vote threshold for consistency gating, and reward weight for reinforcement learning can all be adjusted according to the specific task. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A mirror-oriented visual question-answering method based on subject center coordinate positioning, characterized in that, Let the input sample be x=(I,q,O), where I is the image, q is the question text, and O={(a j,oj )} j=1 m Let a be the set of candidate options. j The letter o represents the option. j The text represents the options, and m represents the number of candidate options; the method includes: S1. Perform structured analysis on x to extract the target subject, target body parts or target objects, spatial relationships, and semantics of candidate options, and then use the reference frame detection function f. ref (q) Obtain the reference frame category r, r∈{F s,Fc,Fu }, where F s F represents the main reference frame. c F represents the camera reference frame. u S2 represents an ambiguous frame of reference; when r=F s Or r=F u Furthermore, when routed to the main reference frame, the main center coordinates are used to locate the path, including through the type routing function f. type (x) yields the problem type t, t∈{H} 2,A4,T }, and in the fixed label set Y corresponding to t t Generate canonical semantic labels ŷ∈Y that are independent of the option letters. t ; S3. Map the canonical semantic label ŷ to the option letter â in the current candidate options through deterministic projection π(ŷ,O), and verify the canonical semantic stability under option exchange or rotation transformation through consistency gating G. S4. When r=F c The compatibility and safety path is executed at the time, and the original visual language model is used to obtain the answer a. raw And select a through non-degenerate routing raw As a prediction result; S5. Output the final option letter a* determined by S2 to S4, and record the processing trajectory T=(r,t,ŷ,π,G,route) as a traceable basis.

2. The method according to claim 1, characterized in that, The reference frame detection function f ref (q) includes: configuring the camera reference frame mode set P c With the main reference frame model set P s ; Compare the question text q with P respectively c and P s Perform a match; when q hits P c Output r=F c When q hits P s Output r=F s When q does not hit P c P was also not hit. s Output r=F u It defaults to entering the main body's center coordinate positioning path and retains the rollback risk marker.

3. The method according to claim 1, characterized in that, The type of routing function f type (x) Determine the question type t based on the target subject category, target body part or target object, and the number of candidate options m; when the target subject is a human body and m=2, t=H2 and Y t= {Left, Right}; When the target is an animal and m=4, t=A4 and Y t= {Left front, right front, left back, right back}; When the sample is a text judgment problem, t=T and Y t= {whether}.

4. The method according to claim 1, characterized in that, The canonical semantic label ŷ is solved by a dedicated solver g. θ (I,q,O,t) are generated, and ŷ∈Y are satisfied. t The dedicated solver is constrained to output the standard semantic label first and then the option letter, making the visual semantic judgment independent of the candidate option letter 'a'. j And their arrangement order; the dedicated solver includes at least one of a conservative human two-choice adapter for human two-choice questions and a semantic verifier for animal four-choice questions.

5. The method according to claim 1, characterized in that, The deterministic projection π(ŷ,O) includes: for each candidate option text o j Execute the normalization function n(o) j Normalization operations include removing option letter prefixes, case normalization, filtering non-semantic characters, and mapping synonyms; when there exists a unique j such that n(o j When ) = n(ŷ), the output is π(ŷ,O) = a j When there is no matching item or multiple conflicting matching items, the projection state is recorded as failed and the rollback strategy of consistency gating is triggered.

6. The method according to claim 1, characterized in that, The consistency gating G includes: constructing K option permutation variants O for the same sample. (1) ,...,O (K) Two-choice samples use the A / B exchange variant, and four-choice samples use the candidate option round transformation; these yield the canonical semantic tag set {ŷ (k) } k=1 K When there exists a unique mode y in the set. mode And its vote count (y) mode When the value is not lower than the threshold τ, the output of G passes and adopts π(y). mode,O The result is used as the prediction result; otherwise, the output of G is rejected and the result is reverted to the original visual language model result or the preset robust branch result.

7. The method according to claim 1, characterized in that, The compatibility and security path uses the non-degradable routing function R. safe (r,a raw,asccg When r=F c At that time, R safe Select the original visual language model result a raw Result a of not selecting the main body center coordinate positioning path sccg To avoid misapplying the subject reference frame judgment logic to camera reference frame samples; and to record the selection of a raw The reason for the routing.

8. The method according to claim 4, characterized in that, The dedicated solver g θ This can be achieved through a training process that combines supervised fine-tuning with group-based policy optimization; the supervised fine-tuning phase aims to standardize semantic labels y. i With option letter a i The goal is to enable the model to learn the output format of "first standardizing semantics, then selecting letters". In the group relative policy optimization stage, a group of candidate outputs are sampled for the same input. A total reward R is constructed by weighting the format reward, semantic correctness reward, option arrangement consistency reward and non-degeneracy reward. The parameter θ is updated according to the in-group normalized advantage function. At the same time, the deviation of the current policy from the reference policy is limited by the KL divergence regularization term.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mirror-direction visual question-answering method based on the subject center coordinate positioning as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable by the processor; when the processor executes the computer program, it uses the electronic device as a running vehicle to implement the method of any one of claims 1 to 8.

Images