Exhibition hall natural language positioning method and system based on dual-path spatial semantic field, and storage medium

Abstract

The invention discloses an exhibition hall natural language positioning method and system based on a dual-path spatial semantic field and a storage medium. Performing context coding on the geometric layout of the booth through a layout coding network to form a learnable space base; a text encoder and a posture encoder are used for encoding natural language questions input by a user and audience postures respectively, the natural language questions and the audience postures are fused into unified semantic representation through a fusion network, and anchor semantic codes, relation semantic codes and direct semantic codes used for spatial reasoning are generated; then executing two reasoning paths of an anchor path and a direct path in parallel; and finally, fusing the outputs of the two paths in a probability domain to obtain booth probability distribution, and synchronously outputting a target booth ranking, anchor point interpretation information and a continuous thermodynamic diagram visualization result. According to the method, joint modeling is carried out on natural language questions, audience postures and geometric layout of booths in an exhibition hall scene, so that directional, reference and range question positioning is realized.

Description

Technical Field

[0001] This invention belongs to the field of smart exhibition hall navigation and indoor spatial semantic understanding technology, specifically involving a natural language localization method for exhibition halls. Background Technology

[0002] With the rapid development of smart exhibition halls, digital exhibitions, and indoor navigation systems, the way visitors obtain information within exhibition halls has gradually shifted from traditional paper guidebooks and manual information windows to new human-computer interaction methods such as mobile voice Q&A, interactive navigation screens, augmented reality navigation, and service robots. Unlike general text retrieval or question-and-answer tasks, exhibition hall navigation Q&A has significant spatial sensitivity: the same question, "Which booth is the second one to my right?" will elicit completely different answers depending on the visitor's location and orientation; the same question, "What new energy-related booths are around me?" is better suited to outputting a multi-objective set related to the visitor's distance, direction, and field of vision, presented in the form of a heatmap or list. Correctly answering these questions not only relies on understanding the semantics of the question text but also heavily depends on accurate modeling of the visitor's location, orientation (hereinafter referred to as "posture"), and the geometric layout and relative relationships of the booths.

[0003] In engineering practice, audience location and orientation information can be obtained through technologies such as ultra-wideband positioning, Bluetooth beacons, visual simultaneous localization and mapping, ARKit, or ARCore; booth geometric layout information can be extracted from the organizer's exhibition drawings, CAD or BIM data, map annotations, or on-site surveying. The key to improving tour guide accuracy, covering multiple question types, and achieving interpretable and visual output lies in how to jointly model natural language semantics, audience posture, and booth geometry within a unified reasoning framework.

[0004] Existing exhibition hall navigation and positioning technologies can be broadly categorized into three types. The first type is question-and-answer systems based on keyword matching or pre-set scripts. These systems rely on manually written templates and slot-filling rules, covering a limited number of fixed questions. However, they exhibit poor generalization ability when faced with diverse expressions in natural language and struggle to effectively utilize visitor posture and booth geometry. The second type is methods based on semantic retrieval or sentence vector similarity matching. These methods calculate the semantic similarity between the question and the booth's text description and return the most similar booth. However, since they are essentially still text matching, they often ignore the visitor's location and orientation, as well as the spatial relationships between booths. This results in outputs that, while semantically relevant, are not spatially within the visitor's actual focus direction or distance. The third category is based on regular sectors or discrete grids. These methods answer orientation questions such as "left," "right," "front," and "back" by manually setting angle thresholds, distance thresholds, or grid area divisions. They can work for regular layouts and simple questions, but their robustness is significantly insufficient when faced with irregular display arrangements, reference relationship expressions, ordinal relationship expressions, multi-target range queries, and occlusion. In addition, the maintenance and iteration costs of the rules are high.

[0005] Summarizing the common shortcomings of the three types of schemes mentioned above, we can find that their core problem lies in the failure to establish a joint mapping mechanism among "semantics-pose-geometry". Semantic understanding and spatial computation are separated, making it difficult for existing schemes to simultaneously handle referential and range-based questions in the same method, and also making it difficult to output interpretable anchor information and continuous heatmap visualization results. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, the technical problems to be solved by this invention include at least the following: First, how to jointly model natural language semantics, audience posture, and booth geometric layout within a unified inference framework, so that the output results have clear spatial meaning rather than just text similarity ranking; Second, how to simultaneously cover referential and ordinal questions, as well as range and regional questions, in the same method, avoiding the need to switch different processing rules or models for different question types; Third, how to provide interpretable anchor point information and continuous heatmap output to meet the needs of guided tours and system debugging; Fourth, how to improve adaptability to different exhibition hall layouts, different booth densities, and different exhibition hall scales, while maintaining acceptable inference latency for engineering deployment.

[0007] Based on the technical problem to be solved, this invention proposes a natural language localization method for exhibition halls based on a dual-path spatial semantic field. In the exhibition hall scene, natural language questions, audience postures and booth geometric layout are jointly modeled to achieve directional, referential and range-based question localization.

[0008] The natural language localization method for exhibition halls based on dual-path spatial semantic fields provided by this invention has the following core idea: First, the geometric layout of the booths is context-encoded using a layout coding network to form a learnable spatial base. Second, the user-input natural language questions and audience postures are encoded using a text encoder and a posture encoder, respectively, and then fused into a unified semantic representation using a fusion network, thereby generating anchor semantic codes, relational semantic codes, and direct semantic codes for spatial reasoning. Then, two reasoning paths, the anchor path and the direct path, are executed in parallel, where the anchor path is used to handle referential and ordinal questions, and the direct path is used to handle range and region questions. Finally, the outputs of the two paths are fused in the probability domain to obtain the final booth probability distribution, and the target booth ranking, anchor explanation information, and continuous heatmap visualization results are output simultaneously.

[0009] The overall flow of the method of the present invention is as follows: Figure 1 As shown, the process includes: the input layer receiving booth layout information, audience posture information, and natural language questions; after step S1 (representing step 1) coordinate unification, S2 geometric feature construction and normalization, and S3 layout encoding to form a spatial base; S4~S5 completing text encoding, posture encoding, and semantic fusion to generate three-way semantic codes; S6 calculating relative geometric features; S7 anchor point path and S8 direct path executed in parallel; and S9 fusing in the probability domain and outputting Top-M target booths, anchor point explanations, and continuous heatmaps.

[0010] The natural language localization method for exhibition halls based on dual-path spatial semantic fields provided by this invention comprises the following steps: Step S1: Data Acquisition and Coordinate Unification The system acquires booth layout information, visitor posture information, and user-input natural language questions, and then converts the booth layout information and visitor posture information to the same coordinate system for coordinate unification. The booth layout information includes the geometric boundary description of each booth within the exhibition hall; the visitor posture information includes the visitor's current position coordinates and orientation angle. In a two-dimensional plane coordinate system, the position coordinates can be represented as... The orientation angle can be expressed as , Defined as the direction the audience faces and the coordinate system The angle between the positive directions of the axes; the forward unit vector of the audience is defined according to the stated orientation angle. With lateral unit vector This is to facilitate subsequent calculations of the relative positions between visitors and each booth.

[0011] Step S2: Construction and Normalization of Booth Geometric Features After completing data acquisition and coordinate unification, a geometric feature vector is constructed for each booth, and the geometric feature vector is normalized.

[0012] The exhibition hall has a total of The first booth, the The center coordinates of each booth are Width is The height is The booth area can be calculated from the width and height. and the aspect ratio of the booth In a preferred embodiment, the area is logarithmically transformed to compress the numerical range, denoted as . ,in To prevent small positive numbers from reaching zero in logarithmic operations, the above parameters are combined to construct the first... Geometric feature vectors of individual booths:

[0013] The constant term at the end is used to enhance the expressive power of the linear transformation.

[0014] The purpose of the normalization process is to eliminate differences in size and scale among different pavilions, thereby improving the model's cross-pavilion generalization ability. Specifically, let the width of the pavilion's planar boundary be... Height is Then, the coordinate and dimension parameters can be proportionally normalized:

[0015] For area transformation value The vector is then further standardized to have a mean of zero and a standard deviation of one. The normalized geometric eigenvector is denoted as... This serves as the input for the subsequent layout coding network.

[0016] Step S3, Layout Coding The normalized geometric feature vectors of all booths are fed into the layout encoding network as an input sequence to obtain the booth embedding of each booth and the global layout representation that represents the global layout.

[0017] In a preferred embodiment, the layout coding network employs a Transformer encoder structure, the core of which is a multi-head self-attention mechanism. Normalized geometric eigenvectors of individual booths The organization is fed into the Transformer encoder in sequence. The encoder captures the contextual relationships between booths through a self-attention mechanism, including but not limited to adjacency relationships, row and column arrangement structures, local density distributions, and spatial patterns such as corridor and partition boundaries. The encoder output is a booth embedding matrix:

[0018] Among them, the first row vector That is, the first Individual booth embedding For embedded dimensions, It is the set of real numbers.

[0019] Furthermore, the method of the present invention also obtains a global layout representation from a layout coding network. This is used to characterize the overall layout and structural features of the exhibition hall. The global layout is obtained by adding a global token to the input sequence and taking its corresponding encoder output, or it can be obtained by embedding the booths into a matrix. Obtained by performing average pooling or attention pooling.

[0020] The technical effect of the layout encoding step is that it transforms discrete booth geometric information into a continuous, learnable spatial representation, and through a self-attention mechanism, the embedding of each booth not only includes its own geometric features, but also implicitly includes its relative positional relationship with surrounding booths and layout context information, thereby providing a stable spatial base for subsequent semantic injection and spatial reasoning.

[0021] Step S4, Text Encoding and Pose Encoding After the layout encoding is completed, the natural language questions input by the user and the audience's posture information are encoded separately to obtain the semantic representation of the questions and the semantic representation of the postures.

[0022] Using a text encoder to process natural language questions Text encoding is performed to obtain the semantic vector of the question. :

[0023] The audience's posture information is encoded using a posture encoder to obtain a posture semantic vector. Specifically, the normalized position coordinates of the audience are first... Trigonometric representation of the orientation angle Combined into attitude fundamental vectors:

[0024] The attitude basis vector is then input into the attitude encoder to obtain the attitude semantic vector. :

[0025] The purpose of expressing the orientation angle in trigonometric function form is to avoid the angle value being... and The discontinuity at the location allows orientation information to be fused and learned with position information in the same vector space.

[0026] Step S5: Semantic Fusion and Multi-path Semantic Code Generation In obtaining the semantic representation of the question With gesture semantic representation Subsequently, the two are fused into a unified fused semantic representation through a fusion network. Furthermore, the fused semantic representation generates a three-way semantic code.

[0027] The input to the fusion network includes question semantic representations. Posture semantic representation and global layout representation The fusion method employs vector concatenation followed by multilayer perceptron transformation, gating fusion mechanism, or cross-attention mechanism. The output of the fusion network is a fused semantic representation:

[0028] in It also encodes information from three aspects: "what users ask", "where the audience is", and "where the audience is looking".

[0029] In obtaining fused semantic representation Subsequently, anchor semantic codes are generated using three independent linear projection heads. Relational semantic codes Compared with direct semantic codes :

[0030] The anchor semantic code Used to identify reference sections that may be involved in the question in subsequent anchor path; the relational semantic code Used to infer the spatial relationship between the reference booth and the target booth under anchor point conditions; the direct semantic code Used to infer range or region probability fields in direct paths without relying on explicit references.

[0031] The technical advantage of multi-path semantic code design is that it enables the semantic representation of the same question to be used differently according to different reasoning paths, thereby taking into account the processing needs of reference-based questions and scope-based questions.

[0032] like Figure 2 As shown, the question The semantic vector of the question is obtained through the text encoder. The audience's posture is used to construct a posture semantic vector and then processed by a posture encoder to obtain a posture semantic vector. ; , Representation of global layout Inputting the fusion network yields the fused semantic representation ; Then through three independent projection heads , , Generate anchor semantic codes respectively Relational semantic codes Compared with direct semantic codes .

[0033] Step S6: Calculation of relative geometric features Before performing spatial reasoning, the relative geometric features between the audience and each booth are calculated so that subsequent anchor path and direct path can utilize spatial relationship information.

[0034] For the For each booth, the vector pointing from the viewer to the center of that booth is defined as:

[0035] in For audience seating, The center of the booth is defined by the vector. The Euclidean distance between the visitor and the booth can be calculated from this vector:

[0036] Using the forward unit vector defined in step S1 With lateral unit vector Calculate the forward and lateral components between the audience and the booth:

[0037] The forward component A positive value indicates the booth is located in front of the audience, and a negative value indicates it is located behind the audience; the lateral component A positive value indicates the booth is located to the right of the audience, and a negative value indicates it is located to the left. Simultaneously, the relative azimuth angle is calculated:

[0038] Used to describe in more detail the angular offset of the booth relative to the direction the audience is facing.

[0039] Combining the above calculation results, a relative feature vector of audience-booth is constructed:

[0040] The relative feature vector quantifies information such as "where the audience is, which direction they are facing, where the booth is located, and how big the booth is" into learnable numerical features for use by the subsequent inference network.

[0041] In the second stage of the anchor point path (i.e., positioning starting from the anchor point), it is also necessary to calculate the relative geometric features between the anchor point booth and each candidate target booth. Let the anchor point booth be the... One booth, target booth is number one. For each booth, an anchor point-booth vector can be defined. Similarly, distance, direction components, relative azimuth, and optional adjacency features are calculated and combined into an anchor-extension relative feature vector. .

[0042] Step S7, Anchor Path Reasoning The method of this invention designs anchor paths to handle referential and ordinal questions. The anchor path consists of two stages: an anchor selection stage and an anchor condition relationship reasoning stage.

[0043] During the anchor point selection phase, the probability of each booth serving as a reference anchor point is calculated. Specifically, the booth embedding is first performed on each booth. Representation of global layout Perform a hybrid transformation to obtain the anchor query vector:

[0044] Then calculate the anchor point score:

[0045] in Represents the vector dot product. For temperature coefficient, This is a multilayer perceptron. Finally, the anchor point scores for all booths are normalized using softmax to obtain the anchor point probability distribution:

[0046] Select the anchor point with the highest probability from the anchor point probability distribution. Individual booths as a set of candidate anchor points and in the set The probabilities of each anchor point are renormalized to obtain the anchor point weights. .

[0047] In the anchor condition relation reasoning phase, for each candidate anchor... Calculate the probability distribution of each booth becoming the target under the anchor point condition. Specifically, first calculate the anchor point-booth relative feature vector. Then embed the anchor point display. Candidate booth embedding Representation of global layout Perform a hybrid transformation to obtain the relation query vector:

[0048] Next, calculate the relationship score:

[0049] Then, softmax normalization is performed on all candidate booths to obtain anchor points. Target probability distribution under the given conditions:

[0050] Finally, the method of this invention weights and mixes the target probability distributions under each candidate anchor point condition according to the anchor point weights to obtain the output probability distribution of the anchor point path:

[0051] The technical effect of the anchor point path is that when the question contains reference or ordinal expressions, the model can first identify possible reference booths, and then reason based on the relative relationship between the reference booths and the target booths, thereby improving the processing stability of expressions such as "second", "next", "opposite", "right next to", and can output anchor point explanation information for navigation display and system debugging.

[0052] like Figure 3 As shown, the anchor point path is divided into two stages: in the anchor point selection stage, the booth embedding... With global representation The anchor query vector is obtained by concatenation and multilayer perceptron transformation. Combined with anchor semantic code Relative characteristics of the audience and the booth Calculate the anchor point score and obtain the anchor point probability distribution using softmax. Select Top- Construct a candidate anchor set The anchor weights are then normalized. In the relational reasoning stage, for each anchor point Embed the anchor point Candidate booth embedding With global representation The relation query vector is obtained by concatenating and transforming the data using a multilayer perceptron. Combined with relational semantic codes Relative features of anchor point-booth Calculate the relationship scores and obtain the conditional probability distribution using softmax. The weighted mixture based on anchor point weights is obtained The direct path method similarly transforms the booth embedding and global representation by concatenating them, and then combines them with direct semantic codes. Output The two paths are fused in S9 according to the fusion coefficient. The final probability distribution is obtained by fusion. and output Top- Target booth, anchor point explanation and continuous heat map.

[0053] Step S8, Direct Path Reasoning The method of this invention designs a direct path for handling range-based and region-based questions. This direct path does not explicitly rely on anchor points, but rather infers the target probability distribution directly based on audience posture and booth relationships.

[0054] Specifically, firstly, the booth embedding for each exhibition booth... Representation of global layout Perform a hybrid transformation to obtain the direct query vector:

[0055] Then calculate the direct score:

[0056] in For temperature coefficient, This is a multilayer perceptron. Finally, softmax normalization is applied to the direct scores of all booths to obtain the output probability distribution of the direct path:

[0057] The technical advantage of the direct path is that when the question expresses a range or region semantic such as "around", "nearby", "a area in front", "closest to me", the model can learn a probability field centered on the audience's position and decaying in a specific direction or distance, thereby supporting multi-target output and continuous thermal visualization without manually setting sector angle thresholds or distance thresholds.

[0058] Step S9: Probability Fusion and Result Output After outputting the probability distributions for the anchor path and the direct path respectively, the outputs of the two paths are fused in the probability domain to obtain the final booth probability distribution.

[0059] The probability fusion uses a weighted summation method:

[0060] in This refers to the fusion coefficient. Hyperparameters can be set to fixed values ​​or can be represented by fusion semantics. Dynamic prediction:

[0061] in This is the sigmoid function. Using dynamic prediction allows the model to adaptively adjust the weights of the two paths based on the semantic features of the question: when the question is biased towards reference or ordinal representation, The value tends to be larger, and the anchor point path dominates; when the question leans towards a range or region expression. The trend is toward smaller values, with the direct path dominating.

[0062] Obtaining the final booth probability distribution Subsequently, the method of the present invention outputs the following results: First, the output with the highest probability. Each target booth and its corresponding confidence level forms a Top- Target booth list; second, the most probable output anchor path. Third, output a continuous heatmap by identifying candidate anchor points and their corresponding weights. It is used to visualize the spatial form of the final probability distribution on the exhibition hall floor plan.

[0063] The continuous heatmap is generated by kernel function superposition:

[0064] in The scaling parameter of the kernel function can be adjusted according to booth density. The output heatmap is further overlaid with booth boundaries, visitor location points, and directional arrows, and labeled Top- The booth number or name forms a visual result that can be directly used for display on the guide screen or rendered on mobile devices.

[0065] like Figure 4 As shown, the process of generating the continuous heatmap is as follows: [The heatmap is generated from the center of each booth.] Centered on the kernel function, with the final probability As weight, with The scale parameter is obtained through the kernel function superposition formula. Generate a continuous thermal field; then, superimpose the booth boundary rectangle, visitor location points, and directional arrows onto the thermal field, and label it Top- The booth number or name; the final output is a PNG / PDF image or front-end renderable layer data for use in the navigation screen or mobile rendering.

[0066] Furthermore, as a preferred embodiment, in step S1, the geometric boundary of each booth is represented as a rectangle. In this case, the geometric boundary description includes the two-dimensional coordinates of the booth center, the width of the booth, and its height. For non-rectangular booths, equivalent representations can be achieved using parameters such as the centroid coordinates of the polygon boundary, the area of ​​the polygon, and the direction of the polygon's principal axis.

[0067] Furthermore, in step S1, the booth layout information may optionally include a text description, category label, or attribute vector for each booth, so as to introduce booth attribute constraints in subsequent reasoning.

[0068] Furthermore, in practical applications, the booth layout data in step S1 may come from exhibition drawings or a CAD system, while the audience posture data may come from an indoor positioning system or a visual positioning module. The coordinate origin, scale, or rotation angle of the two may differ. Through coordinate registration, scale calibration, and rotation and translation transformations, the two can be unified into the same exhibition hall plane coordinate system.

[0069] Furthermore, the text encoder in step S4 is a pre-trained Chinese language model, sentence vector encoder, or lightweight text encoding network; the output of the text encoder is a sentence-level representation vector of the question, the dimension of which matches the input dimension of the subsequent fusion network.

[0070] Furthermore, the attitude encoder described in step S4 can employ a multilayer perceptron or a position coding network to map the low-dimensional attitude basis vectors to a high-dimensional attitude semantic vector with the same dimension as the question semantic vector. .

[0071] To enable those skilled in the art to fully implement the method of this invention, the following describes optional implementation details for implementing the complete training and deployment of this method.

[0072] The training data for the method described in this invention can be obtained in the following ways: First, manual annotation, that is, labeling the questions with the corresponding target booth numbers, and additionally labeling anchor booth numbers for reference questions; Second, automatic synthesis, that is, randomly sampling the audience positions and orientations based on the known booth layout, and generating questions such as "the nth one in front of the right", "to the right of A", "what's around" and their corresponding labels according to the preset sentence template; Third, log-based weak supervision, that is, constructing weakly labeled data using user interaction logs, click behaviors, or dwell hot zones.

[0073] The training objective of the method described in this invention may include the following loss function: Target booth classification loss:

[0074] in This is the actual target booth. Anchor point monitoring loss (if indicated):

[0075] in This is a real anchor point display. Direct path auxiliary loss:

[0076] The total loss is:

[0077] in and This is the loss weighting coefficient.

[0078] During the deployment phase, the output of the layout coding step (standby embedding) With global representation It can be calculated and cached offline when the exhibition hall is updated; when reasoning online, only the question and posture need to be encoded and forward reasoning is performed once to obtain the output result, thus meeting the needs of low-latency real-time tour guidance.

[0079] The present invention also provides a natural language positioning system for exhibition halls, comprising multiple modules for performing the method described in any one of claims 1 to 7.

[0080] The present invention also provides a natural language positioning device for exhibition halls, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of any one of claims 1-7.

[0081] The present invention also provides a computer storage medium, characterized in that it stores one or more computer programs, which are executed by a processor to implement the method according to any one of claims 1 to 7.

[0082] Compared with the prior art, the method of the present invention has the following beneficial effects: First, this invention uses a layout coding network to contextually encode the booth's geometric layout to form a spatial base, and then uses a fusion network to integrate the semantics of the question with the audience's posture into a unified representation, achieving joint modeling of "semantics-posture-geometry". This technique solves the spatial mismatch problem caused by the lack of spatial information in existing semantic retrieval methods, ensuring that the output results are consistent with the audience's actual position and orientation.

[0083] Second, this invention designs two parallel reasoning paths: an anchor path and a direct path. The anchor path processes referential and ordinal questions through two-stage reasoning, while the direct path processes range-based and region-based questions through probability field modeling. The two paths are fused and output in the probability domain. This technique solves the problem of incomplete coverage of complex question types in existing rule sector methods, enabling the same method to accommodate multiple question expression forms.

[0084] Third, this invention outputs candidate anchor points and their weights along the anchor point path, and a continuous heatmap in the final result, making the positioning results interpretable and visualizeable. This technique solves the problem of uninterpretable output from existing end-to-end models, facilitating navigation screen display, user understanding, and system debugging.

[0085] Fourth, this invention enables the method to adapt to different exhibition hall sizes and irregular booth arrangements through geometric normalization and self-attention layout representation, reducing the maintenance and iteration costs of manual rules.

[0086] Fifth, the layout encoding results of this invention can be cached offline, and online inference only requires one forward calculation, which meets the requirements of low-latency real-time navigation and facilitates engineering deployment in scenarios such as navigation screens, mobile terminals or service robots. Attached Figure Description

[0087] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0088] This diagram illustrates the process starting from the input layer (booth layout information, visitor posture information, and natural language questions), sequentially proceeding through steps S1 (data acquisition and coordinate unification), S2 (geometric feature construction and normalization), and S3 (layout encoding, outputting the booth embedding matrix). With global representation Steps S4-S5: Text encoding, gesture encoding, and semantic fusion (generating anchor semantic codes) Relational semantic codes Compared with direct semantic codes Step S6: Calculation of relative geometric features; Step S7: Parallel inference of anchor path and direct path; Step S9: Probabilistic fusion and output of the complete data flow and module relationship.

[0089] Figure 2 A flowchart illustrating the process of text encoding, pose encoding, and fusion to generate multi-path semantic codes.

[0090] The image shows a question. The semantic vector of the question is obtained through the text encoder. The audience's posture information is used to construct posture semantic vectors and then combined with a posture encoder to obtain posture semantic vectors. , , Representation of global layout Inputting the fusion network yields the fused semantic representation , Then through three independent projection heads , , Generate anchor semantic codes respectively. Relational semantic codes Compared with direct semantic codes The process.

[0091] Figure 3 This is a schematic diagram illustrating the reasoning process involving anchor paths, direct paths, and probability fusion.

[0092] This diagram illustrates the two stages of the anchor point path: the anchor point selection stage, and the booth embedding stage. With global representation The anchor query vector is obtained by concatenation and multilayer perceptron transformation. Combined with anchor semantic code Relative characteristics of the audience and the booth Calculate the anchor point score and obtain the anchor point probability distribution using softmax. Select the one with the highest probability. Individual booths constitute a set of candidate anchor points The anchor weights are then normalized. In the relational reasoning stage, for each anchor point Embed the anchor point Candidate booth embedding With global representation The relation query vector is obtained by concatenating and transforming the data using a multilayer perceptron. Combined with relational semantic codes Relative features of anchor point-booth Calculate the relationship score and obtain the conditional probability distribution using softmax. The anchor point path output is obtained by weighted mixing according to the anchor point weight. The direct path method similarly transforms the booth embedding and global representation by concatenating them, and then combines them with direct semantic codes. Relative characteristics output of the audience and the booth The probabilistic fusion stage is based on the fusion coefficient. The final probability distribution is obtained by weighted summation of the outputs from the two paths. and output Top- List of target booths, anchor point explanations, and continuous heatmap.

[0093] Figure 4 This is a schematic diagram illustrating the generation and overlay visualization output of heatmaps.

[0094] This diagram illustrates the process of generating a continuous heatmap: centered around each booth. Centered on the kernel function, with the final probability As weight, with The scale parameter is obtained through the kernel function superposition formula. Generate a continuous thermal field; then, superimpose the booth boundary rectangle, visitor location points, and directional arrows onto the thermal field, and label it Top- The booth number or name; the final output is a PNG / PDF image or front-end renderable layer data for use in the navigation screen or mobile rendering. Detailed Implementation

[0095] The technical solution of the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be noted that the following embodiments are only used to explain the technical solution of the present invention and should not be construed as limiting the scope of protection of the present invention; equivalent substitutions made by those skilled in the art in terms of network structure, feature combination, fusion method and output form without departing from the core idea of ​​the present invention should all fall within the scope of protection of the present invention.

[0096] Example 1: Processing of Directional and Ordinal Questions This embodiment uses the question "Which company is the second booth from the right?" as an example to illustrate the complete execution process of the method of the present invention.

[0097] First, we need to set up the layout. Assume the exhibition hall's floor plan dimensions are [width value missing]. meters, height The booths are arranged in a 6x6 grid, with each booth measuring [dimensions missing]. rice Meters. The center coordinates of each booth are generated according to the following rules: for row index With column index , No. The center coordinates of each booth are Meters. Booth number defined as That is, the booths are numbered sequentially from the bottom left corner, with a total of 36 booths.

[0098] Then, define the audience's posture and the question. Assume the audience is currently located at coordinates... At meters, facing the angle Radius, referring to the northeast direction. The user-input question is... Which company has the second booth from the right? Next, steps S1 to S9 are executed. In step S2, geometric feature vectors are constructed and normalized for each of the 36 booths. In step S3, the normalized geometric feature sequence is input into the layout coding network to obtain the booth embedding matrix. With global representation In step S4, the question is text-encoded to obtain... The pose is encoded to obtain In step S5, fusion , and Obtain the fused semantic representation And generate three semantic codes. , , .

[0099] In step S6, the relative geometric features between the audience and each booth are calculated. (Based on booth...) (Row Index) Column index center coordinates For example, calculate:

[0100] The distance is:

[0101] According to the orientation angle Forward unit vector:

[0102] Lateral unit vector: .

[0103] The forward and lateral components are as follows:

[0104]

[0105] Since both the forward and lateral components are positive, booth 21 is located in the area to the right and in front of the audience. Similarly, the relative characteristics of other booths can be calculated. Booths located in the area to the right and in front also include... (center , rice, rice), (center , rice, (meters), etc.

[0106] In the anchor point selection stage of step S7, the model generates an anchor point probability distribution for each booth based on the ordinal semantics of "second" in the question and the relative characteristics of the audience and booth. Typically, booths located to the right front and at a suitable distance (such as...) are considered anchor points. , They will obtain a higher anchor point probability because they are key reference nodes in the "right-front sequence". Assume... Candidate anchor set The corresponding anchor weights are respectively (Example value).

[0107] In the relational reasoning stage of step S7, the model calculates the probability distribution of the relationship between each candidate anchor and each booth. (Relational semantic code) It encodes the relative semantics of "second", combined with anchor-slot relative features. The model learned the spatial pattern of "being in the second position after being sorted along the forward direction".

[0108] In step S8, the probability distribution of the direct path output covering the right front region is calculated. This serves as a supplement to the anchor path.

[0109] In step S9, assuming the fusion coefficient The final probability is:

[0110] Output Top- Target booth list, for example (Example numerical value). Also output anchor point explanation information. It generates a continuous heatmap overlaid with arrows indicating booth boundaries and visitor locations.

[0111] Through the above process, the method of the present invention can correctly identify the target corresponding to "the second booth from the right front" and provide anchor point explanation and visualization results.

[0112] Example 2: Processing of Reference and Adjacent Questions This embodiment uses the question "Who is right next to booth A on the right side?" as an example to illustrate the processing of explicit reference questions by the method of the present invention.

[0113] The layout settings of Example 1 are used. Assume booth A is... (Center coordinates) ), its adjacent booth on the right is (Center coordinates) The question entered by the user is... = Who is standing right next to booth A? The audience posture can be any value, because the answer to this question depends mainly on the reference relationship rather than the audience's position.

[0114] In the text encoding process of step S4, the "A-side" in the question is encoded as a strong anchor semantic, making the anchor semantic code... A higher response is generated when calculating the inner product with the embedding of booth 21. In the anchor selection phase of step S7, the anchor probability of booth 21... Reaching peak, entering Top- Candidate anchor set And its weight near .

[0115] In the relational reasoning stage of step S7, the model calculates the anchor-booth relative characteristics between anchor point 21 and each booth. For booth 22:

[0116] Booth 22 is located directly to the right of booth 21, at a distance of 5 meters. Given the current layout with a 5-meter spacing between booths, booths 22 and 21 are considered adjacent. (Relationship semantic code) It encodes the semantics of "right side next to", combined with Based on the directional and distance features, the model outputs a high probability at booth 22.

[0117] In step S9, since the anchor path already has a clearly defined target, the contribution of the direct path is relatively small. The final output Top-1 target display position is... Anchor point explanation information This indicates that the model uses booth 21 as a reference anchor point for inference.

[0118] Through the above process, the method of the present invention can correctly handle explicit reference questions and provide clear anchor interpretations.

[0119] Example 3: Processing of Scope and Multi-Target Questions This embodiment uses the question "What new energy-related booths are around me?" as an example to illustrate the processing process of the method of the present invention for range-based multi-objective questions.

[0120] The layout settings of Example 1 are used, and attribute labels are configured for some booths. Assume that the new energy-related booths are... The remaining booths were for non-new energy categories. The user-entered question was... = "What new energy-related booths are around me?" (The audience is located at coordinates...) Meters away.

[0121] In step S5, the semantic representation is fused. It encodes the scope of multi-target semantics, including "around" and "what is". Since the question does not involve explicit references or ordinal expressions, the fusion coefficient... tending towards smaller values ​​(e.g.) ), making the direct path It dominates in the final probability.

[0122] In step S8, the direct path is determined based on the direct semantic code. Relative characteristics of the audience and the booth Output probability distribution. The semantics of "surrounding" are learned by the model as a distance-decaying probability field centered on the audience's location, with booths closer to the audience having a higher probability.

[0123] In step S9, the final probability is:

[0124] Because the new energy-related booths are distributed in different directions around the audience, the heat map shows multiple peaks. (Top-output) List of booths related to new energy, for example (Example value).

[0125] Through the above process, the method of the present invention can correctly handle range-based multi-objective questions and visually display the distribution of targets through heatmaps.

[0126] Boundary cases and variations To enhance the feasibility and generalizability of the method of the present invention, several boundary cases and variations are described below.

[0127] First, handling non-rectangular booths. When the booth boundary is a polygon rather than a rectangle, the centroid of the polygon can be used as the center coordinate of the booth. Using polygon area Replace width with parameters such as spindle direction With height The remaining steps remain unchanged.

[0128] Second, the handling of multi-story or three-dimensional exhibition halls. When an exhibition hall has multiple floors or differences in height, the coordinates can be transformed from two-dimensional to three-dimensional. Expand to three dimensions , will head from a single yaw angle Expanded to a combination of yaw and pitch The relative features are then extended into a three-dimensional form.

[0129] Third, handling orientation uncertainty. When there is significant noise in the measurement of audience orientation, the orientation confidence level can be used as an input feature to the attitude encoder, or the orientation angle can be adjusted. To improve stability, we perform small-scale sampling and fuse multiple inference results.

[0130] Fourth, handling occlusion and visibility. When considering the occlusion relationship between booths, it is advisable to consider the relative features... Add channel map distance or visible field approximation features to enhance the processing capabilities of expressions such as "what I can see" or "the row in front".

[0131] Fifth, processing of multi-objective and threshold outputs. For questions like "What are some", in addition to outputting Top-... In addition to the list, it can also output probabilities greater than a set threshold. All booth collection To adapt to multi-target output requirements.

[0132] Symbol Explanation To facilitate understanding and reproduction of the technical solutions of this invention by those skilled in the art, the main symbols used in this specification are summarized and explained below.

[0133] This indicates the total number of booths in the exhibition hall; This represents the booth index, with a value range of [value range missing]. to ; This indicates the anchor index.

[0134] Indicates the first The center coordinates of each booth; Indicates the first The width of each booth; Indicates the first The height of each booth; Indicates the first The area of ​​each booth; Indicates the first The aspect ratio of each booth; Represents the logarithmic transformation value of the area.

[0135] Indicates the coordinates of the audience's location; Indicates the angle of the audience's orientation; Represents the forward unit vector of the audience; This represents the lateral unit vector of the audience.

[0136] Indicates the first Geometric feature vectors of each booth; This represents the normalized geometric eigenvector.

[0137] Represents the booth embedding matrix; Indicates the first Booth embedding for individual exhibition booths; This represents the global layout. Indicates the embedding dimension.

[0138] Represents the semantic vector of a question; Represents a pose semantic vector; This represents a fusion semantic representation; Represents anchor semantic code; Represents relational semantic codes; This represents a direct semantic code.

[0139] Indicates the audience and the first The relative feature vectors of visitors to each booth; Indicates anchor point With the Anchor points between booths—relative feature vectors of booths.

[0140] This represents the probability distribution of the booth output from the anchor path; This represents the booth probability distribution of the direct path output; This represents the probability distribution of the final merged booths.

[0141] , , These represent the temperature coefficients in anchor point scoring, relationship scoring, and direct scoring, respectively. Represents the probability fusion coefficient; The scale parameter representing the kernel function of the heatmap; Indicates the number of candidate anchor points; This indicates the target number of booths to be output.

Claims

1. A natural language localization method for exhibition halls based on dual-path spatial semantic fields, characterized in that, Includes the following steps: Step S1: Data Acquisition and Coordinate Unification The system acquires booth layout information, visitor posture information, and user-input natural language questions. It then converts the booth layout information and visitor posture information to the same coordinate system for coordinate unification. The booth layout information includes the geometric boundary description of each booth within the exhibition hall. The visitor posture information includes the visitor's current position coordinates and orientation angle. In a two-dimensional plane coordinate system, the position coordinates can be represented as... The orientation angle can be expressed as ,in Defined as the direction the audience faces and the coordinate system The angle between the positive directions of the axes; the forward unit vector of the audience is further defined based on the stated orientation angle. With lateral unit vector This is to facilitate subsequent calculations of the relative positions between visitors and each booth; Step S2: Construction and Normalization of Booth Geometric Features For each booth, a geometric feature vector is constructed, and the geometric feature vector is then normalized. The exhibition hall has a total of The first booth, the The center coordinates of each booth are Width is The height is The booth area can be calculated from the width and height. and the aspect ratio of the booth To compress the numerical range by taking the logarithmic transformation of the area, denoted as . ,in To prevent small positive numbers from having a zero value in logarithmic operations; By combining the above parameters, the first... Geometric feature vectors of individual booths: , The constant term at the end is used to enhance the expressive power of the linear transformation; Let the width of the exhibition hall's plan boundary be Height is The coordinate and dimension parameters are proportionally normalized: , For area transformation value Further standardization is performed to make its mean zero and standard deviation one; the normalized geometric eigenvector is denoted as... , as input to the subsequent layout coding network; Step S3, Layout Coding The normalized geometric feature vectors of all booths are fed into the layout coding network as an input sequence to obtain the booth embedding of each booth and the global layout representation representing the global layout. The layout coding network adopts a Transformer encoder structure; Normalized geometric eigenvectors of individual booths The organization takes a sequence as input to a Transformer encoder. The encoder captures the contextual relationships between booths through a self-attention mechanism, including but not limited to adjacency relationships, row and column arrangement structures, local density distributions, and spatial patterns such as corridor and partition boundaries. The encoder output is a booth embedding matrix. , Among them, the first row vector That is, the first Individual booth embedding For the embedded dimension; Obtain the global layout representation from the layout coding network. This is used to characterize the overall layout and structural features of the exhibition hall; the global layout is obtained by adding a global token to the input sequence and taking its corresponding encoder output, or by embedding the booths into a matrix. Obtained by performing average pooling or attention pooling; Step S4, Text Encoding and Pose Encoding After the layout encoding is completed, the natural language questions input by the user and the audience's posture information are encoded separately to obtain the semantic representation of the questions and the semantic representation of the postures. Using a text encoder to process natural language questions Text encoding is performed to obtain the semantic vector of the question. : , Normalized position coordinates of the audience Trigonometric representation of the orientation angle Combined into attitude fundamental vectors: , The attitude basis vector is then input into the attitude encoder to obtain the attitude semantic vector. : , Step S5: Semantic Fusion and Multi-path Semantic Code Generation In obtaining the semantic representation of the question With gesture semantic representation Subsequently, the two are fused into a unified fused semantic representation through a fusion network. The three-way semantic code is generated from the fused semantic representation: The input to the fusion network includes question semantic representations. Posture semantic representation and global layout representation The fusion method employs vector concatenation followed by multilayer perceptron transformation, gating fusion mechanism, or cross-attention mechanism; the output of the fusion network is a fused semantic representation. , in It also encodes information from three aspects: "what users ask", "where the audience is", and "where the audience is looking". Obtain fused semantic representation Subsequently, anchor semantic codes are generated using three independent linear projection heads. Relational semantic codes Compared with direct semantic codes : , The anchor semantic code Used to identify reference sections that may be involved in the question in subsequent anchor path; the relational semantic code Used to infer the spatial relationship between the reference booth and the target booth under anchor point conditions; the direct semantic code Used to infer range or region probability fields in direct paths without relying on explicit references; Step S6: Calculation of relative geometric features For the For each booth, the vector pointing from the viewer to the center of that booth is defined as: , in For audience seating, Calculate the Euclidean distance between the visitor and the booth, centered on the booth: , Using the forward unit vector defined in step S1 With lateral unit vector Calculate the forward and lateral components between the audience and the booth: , The forward component A positive value indicates the booth is located in front of the audience, and a negative value indicates it is located behind the audience; the lateral component A positive value indicates the booth is located to the right of the audience, and a negative value indicates it is located to the left of the audience; simultaneously, the relative azimuth angle is calculated: , Used to describe in more detail the angular offset of the booth relative to the direction the audience is facing; Combining the above calculation results, a relative feature vector of audience-booth is constructed: , Then calculate the relative geometric features between the anchor booth and each candidate target booth: Let the anchor booth be the i-th... One booth, target booth is number one. For each booth, an anchor point-booth vector can be defined. Similarly, distance, direction components, relative azimuth, and optional adjacency features are calculated and combined into an anchor-extension relative feature vector. ; Step S7, Anchor Path Reasoning During the anchor point selection phase, the probability of each booth serving as a reference anchor point is calculated. First, the booth embedding is performed for each booth. Representation of global layout Perform a hybrid transformation to obtain the anchor query vector: , Then calculate the anchor point score: , in Represents the vector dot product. For temperature coefficient, The algorithm is a multilayer perceptron; finally, the anchor point scores of all booths are normalized using softmax to obtain the anchor point probability distribution: , Select the anchor point with the highest probability from the anchor point probability distribution. Individual booths as a set of candidate anchor points and in the set The probabilities of each anchor point are renormalized to obtain the anchor point weights. ; In the anchor condition relation reasoning phase, for each candidate anchor... To calculate the probability distribution of each booth becoming the target under the anchor point condition, first calculate the anchor point-booth relative feature vector. Then embed the anchor point display. Candidate booth embedding Representation of global layout Perform a hybrid transformation to obtain the relation query vector: , Next, calculate the relationship score: , Then, softmax normalization is performed on all candidate booths to obtain anchor points. Target probability distribution under the given conditions: , Finally, the target probability distributions under each candidate anchor point condition are weighted and mixed according to the anchor point weights to obtain the output probability distribution of the anchor point path: , Step S8, Direct Path Reasoning First, embed the booth design for each exhibition booth. Representation of global layout Perform a hybrid transformation to obtain the direct query vector: , Then calculate the direct score: , in For temperature coefficient, For a multilayer perceptron; the direct scores of all booths are normalized using softmax to obtain the output probability distribution of the direct path: 。 2. The method according to claim 1, characterized in that, In step S1, the geometric boundary of each booth is represented as a rectangle. The description of the geometric boundary includes the two-dimensional coordinates of the booth center, the width and height of the booth. For non-rectangular booths, equivalent representations can be made using parameters such as the centroid coordinates of the polygon boundary, the area of ​​the polygon, and the direction of the polygon's principal axis. In step S1, the booth layout information also includes one or more of the following: text description, category label, and attribute vector for each booth, so as to introduce booth attribute constraints in subsequent reasoning. In step S1, the booth layout information and the audience posture information are converted to the same coordinate system for coordinate unification through coordinate registration, scale calibration, and rotation and translation transformation.

3. The method according to claim 1, characterized in that, The text encoder in step S4 is a pre-trained Chinese language model, sentence vector encoder, or lightweight text encoding network; the output of the text encoder is a sentence-level representation vector of the question, the dimension of which matches the input dimension of the subsequent fusion network; the pose encoder can use a multilayer perceptron or position encoding network to map the low-dimensional pose basis vector to a high-dimensional pose semantic vector with the same dimension as the question semantic vector. .

4. The method according to claim 1, characterized in that, When the booth boundary is a polygon rather than a rectangle, the centroid of the polygon is used as the coordinate of the booth center. Using polygon area Replace width with parameters such as spindle direction With height The remaining steps remain unchanged.

5. The method according to claim 1, characterized in that, When the exhibition hall has multiple floors or different heights, the coordinate system will be changed from two-dimensional to three-dimensional. Expand to three dimensions , will head from a single yaw angle Expanded to a combination of yaw and pitch The relative features are then extended into a three-dimensional form.

6. When there is significant noise in the measurement of audience orientation, the orientation confidence level can be used as an input feature to the attitude encoder, or the orientation angle can be... To improve stability, we perform small-scale sampling and fuse multiple inference results.

7. When it is necessary to consider the obstruction relationship between booths, it can be done in terms of relative features. Add channel map distance or visible field approximation features to enhance the processing capabilities of expressions such as "what I can see" or "the row in front".

8. A natural language positioning system for exhibition halls, characterized in that, It includes multiple modules for performing the method according to any one of claims 1 to 7.

9. A natural language positioning device for exhibition halls, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1-7.

10. A computer storage medium, characterized in that, It stores one or more computer programs, which are executed by a processor to implement the method of any one of claims 1 to 7.

Images