A Perceptual Analysis Method for Sign Differences in Multi-Contextual Linguistic Landscapes

Abstract

This application provides a method for perceptual analysis of differences in multi-contextual language landscape symbols, including: based on the results of character density increase and the early threshold of language switching, using a convolutional neural network to learn features from the density increase sequence along the scanning direction, and outputting the character arrangement fault morphology at each position; using a gradient boosting decision tree to fuse the character arrangement fault morphology with the language boundary position, generating boundary markers between foreign language areas and Chinese language areas; based on the boundary markers between foreign language areas and Chinese language areas, performing regional language labeling on multi-context signs, and outputting the language recognition results of the signs; based on the sign type and language recognition results, determining the sign keywords that directly convey the function of the place corresponding to the sign type, generating preliminary translations in each language for tourist groups of different languages, and writing them into the translation field of the language landscape comprehensive database.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for perceptual analysis of differences in multi-contextual linguistic landscape symbols. Background Technology

[0002] In the field of linguistic landscape research, exploring the language distribution and recognition of signage text in multilingual contexts is of great significance. This area not only concerns cultural exchange and the presentation of urban image but also directly affects the efficiency and accuracy of public information transmission. With the acceleration of globalization, signs displaying a mix of foreign and local languages ​​are increasingly common in cities. Accurately identifying and labeling the boundaries between different languages ​​has become a crucial issue that cannot be ignored in improving information readability and management efficiency. Current research and practical methods often overlook the complexity of text layout on signs, especially the dynamic changes in machine recognition at the boundaries between different languages. Many methods only focus on the language classification of the overall text content without deeply analyzing the specific patterns of text arrangement at the boundaries, leading to frequent inaccuracies in language classification in practical applications.

[0003] This limitation makes it difficult to identify the correlation between changes in text density at the boundary and the timing of language switching when faced with diverse sign designs. Text density refers to the number of characters per unit area, and on signs, the density of different languages ​​often affects people's judgment of language switching points. Theoretically, if the text density increases at the boundary, the switching point should be easier to detect. However, in reality, if the spacing between the characters on both sides is uneven, the judgment of the switching point becomes blurred. This ambiguity stems from the reliance on the regularity of text arrangement in visual perception; inconsistencies in density and spacing interfere with the accuracy of recognition. For example, on a sign in a city's commercial street, when Chinese and foreign languages ​​are mixed, the Chinese characters may be denser and spaced smaller, while the foreign language characters may be sparser and spaced larger. At the boundary, if the density of the two languages ​​suddenly changes, but the spacing distribution is inconsistent, the recognition system will find it difficult to quickly determine the exact location of the switch from Chinese to foreign language, and may even misidentify some foreign language characters as Chinese, leading to labeling errors.

[0004] Therefore, accurately capturing the combined effects of text density and spacing distribution in the boundary area, and dynamically determining the timing of language switching accordingly, has become a key issue in improving the accuracy of signboard language labeling. Summary of the Invention

[0005] This invention provides a method for perceptual analysis of differences in multi-contextual linguistic landscape symbols, the method comprising: The image acquisition module captures real-world images of multi-context signs, and the positioning module binds geographical coordinates to anchor points on Google Maps and Gaode Maps. The signs are classified according to their appearance features and incorporated into the language landscape database. Optical character recognition is used to extract the outline of character strokes, identify the density of the arrangement and the uniformity of the width of the blank space, and determine the result of increased character density and the early threshold of language switching. A convolutional neural network is used to learn the features of increasing density sequences and output the character arrangement tomographic morphology. Candidate positions in the character arrangement tomographic morphology are screened, and language discrimination is performed to obtain the boundary region recognition result. The character segmentation module divides the stroke groups and verifies them with constraint conditions to determine the language boundary position. A gradient boosting decision tree is used to fuse the character arrangement fault morphology with the language boundary position to generate a boundary marker between the foreign language region and the Chinese region. Based on the boundary markers, the language of the signboard is labeled, and the language recognition result is output; Based on the sign type and language recognition results, retrieve the corresponding sign keywords, generate preliminary translations for each language, and write them into the translation field; The semantics of the initial translation are controlled, background cultural translations are added to the names of signs and attractions with extralinguistic background, translation strategy types are marked, and structured cultural annotation data is formed. Based on the structured cultural annotation data, spelling verification, grammar rule scanning, and semantic comparison of the translated text are performed on the signboard text. Error entries are written into the background information field, and language error status is marked on the map anchor points. The signboard information can be confirmed, corrected, added, and commented on by community users. Users can jump to Google Maps and Amap to view the visualized route with one click.

[0006] Furthermore, the image acquisition module captures multi-context real-world images of the signboard, and the positioning module binds the geographic coordinates to anchor points on Google Maps and Gaode Maps. The signboard types are then classified based on appearance features and incorporated into the comprehensive language landscape database, including: The positioning module synchronously records the latitude and longitude coordinates of the shooting point, and the latitude and longitude coordinates are bound to the coordinate nodes of Google Maps and Gaode Maps to obtain a real-scene image of the sign with dual map anchor points; The appearance features of the signboard are extracted, including the shape of the signboard frame, the material of the base material, the posting method, and the graphic logo of the storefront. The signboard type is determined based on the appearance features and then incorporated into the language landscape database.

[0007] Furthermore, the step of extracting character stroke outlines using optical character recognition, identifying the density of character arrangement and the uniformity of white space width, and determining the result of increased character density and the early threshold for language switching includes: The character outline is delineated based on the connected components of the character stroke contour, and the scanning direction is set along the long side of the sign to obtain the character sequence; The density of character enclosing frames within a unit area is obtained by counting the number of such frames. The uniformity of white space width is obtained by measuring the width of the blank pixels between adjacent frames, calculating the standard deviation, and taking the reciprocal. The character density is increased by averaging the arrangement density using a fixed window length sliding window and multiplying it by the uniformity of the white space width position by position. The difference is then calculated along the scanning direction, and the position where the density increment exceeds a preset threshold is marked as the language switching early threshold.

[0008] Furthermore, the method of using a convolutional neural network to learn the features of the increasing density sequence and outputting the character arrangement tomographic morphology includes: The density-increasing sequence is extracted by extending a preset length to the left and right from the language switching early threshold, and then normalized to the maximum and minimum values ​​and padded with zero values ​​to a fixed length. The one-dimensional convolutional layer, max pooling layer, and fully connected layer of the convolutional neural network process the density increasing sequence in sequence, and output a fault morphology vector containing the density increase magnitude, density decrease magnitude, and fault straightness. The fault morphology vector is classified position by position into abrupt, gradual, or transitional fault morphology to obtain the character arrangement fault morphology.

[0009] Furthermore, the process of filtering candidate positions in the character arrangement tomographic morphology, performing language discrimination to obtain boundary region identification results, segmenting stroke groups by the character segmentation module and verifying them with constraints to determine the language boundary positions includes: The coordinates of the locations located within a preset range near the language switching threshold and accompanied by abrupt or gradual fault patterns are selected to obtain a set of candidate locations. The outlines of the surrounding character strokes are extracted for each candidate location and fed into a random forest classifier. The language of the characters on the left and right sides is determined based on the stroke direction, stroke thickness ratio, and character height-to-width ratio to obtain the boundary region identification result. The character segmentation module reclassifies the connected components in the boundary area along the scanning direction, inserts a segmentation line where the stroke outline height or stroke thickness deviation exceeds a preset difference, and merges them to obtain foreign language stroke groups and Chinese stroke groups. Using the density of the arrangement and the uniformity of the width of the blank space as the constraints, the location where the difference in density of the arrangement on both sides of the junction between the foreign language stroke group and the Chinese stroke group exceeds a preset threshold and the uniformity of the width of the blank space is in the opposite direction is determined as the language boundary location.

[0010] Furthermore, the step of using a gradient boosting decision tree to fuse the character arrangement fault morphology with the language boundary position to generate the boundary marker between the foreign language region and the Chinese region includes: Using the horizontal coordinate of the language boundary as the anchor point, the three components of density increase, density decrease and fault straightness within a preset length to the left and right of the anchor point are extracted and combined with the corresponding morphological labels to obtain the fused feature vector. A decision tree is built through multiple iterations. The leaf nodes output language affiliation scores. After weighted summation, positive values ​​represent Chinese regions and negative values ​​represent foreign language regions. Adjacent positions with opposite signs and crossing a preset zero threshold are determined as boundary points. The boundary marker between the foreign language region and the Chinese region is obtained by embedding segment labels.

[0011] Furthermore, the step of labeling the language of the signboard zones based on the boundary markers and outputting the language recognition result includes: Based on the horizontal coordinates of the endpoints of the segments in the boundary markers, the real-world image of the signboard is divided into several segments along the direction of the signboard, and the start and end ranges of the horizontal coordinates of each segment and the foreign language label or Chinese language label are obtained. Foreign language characters in the foreign language area are matched with foreign language labels based on optical character recognition, and Chinese characters in the Chinese language area are labeled as Chinese. The language recognition results are written into the language field of the language landscape comprehensive database.

[0012] Furthermore, the step of retrieving corresponding signboard keywords based on the signboard type and language identification results, generating preliminary translations for each language, and writing them into the translation field includes: Based on the sign type, the corresponding sign keywords are retrieved from a pre-established venue function keyword database, where the venue function keyword database uses sign type as the key and Chinese sign keywords that directly convey the venue function as the value. For Chinese, English, Russian, Korean, Japanese, French, Spanish and Arabic, neural machine translation is used to translate the signboard keywords into multiple languages ​​with Chinese as the source language, and the preliminary translations in each language are written into the translation field.

[0013] Furthermore, the initial semantic control of the translated text, supplementing the background cultural translations of the superlinguistic background signs and scenic spot names, and labeling the translation strategy types, forming structured cultural annotation data, includes: By comparing the preliminary translation with the pre-established cultural term index, the super-language background signs and scenic spot names that carry cultural connotations beyond the literal meaning are identified and the background cultural translations are supplemented to obtain a multilingual translation set with cultural translations. Each translation in the multilingual translation collection is labeled with a translation strategy type that is either phonetic-preserving, semantic-conversion, or hybrid. The cultural adaptation path is recorded in three segments: the original name, the conversion method, and the target language. The translated name entries, strategy tags, and path descriptions are combined to form a structured entry and stored in the database, thus forming the structured cultural annotation data.

[0014] Furthermore, based on the structured cultural annotation data, spelling verification, grammar rule scanning, and semantic comparison of the translated text are performed on the signboard text. Error entries are identified and written into the background information field. Map anchor points are used to mark language error status. The system also provides community users with the ability to confirm, correct, add, and comment on signboard information, and allows one-click access to Google Maps and Gaode Maps to view visualized routes. The extracted sign text is subjected to spelling verification by dictionary comparison, grammar rule scanning by grammar rule base matching, and semantic comparison of the translated text is performed based on the target language translation in the structured cultural annotation data to obtain the error type, the region, the original text and the suggested correction text, and the error entries are written into the background information field of the coordinate point image. The language error status is marked at the map anchor point according to the error entry, and the correction suggestions submitted by the open community users are written back to the background information field; The map anchor point embeds a one-click jump entry to the visualized route on Google Maps and Amap.

[0015] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a multi-contextual language landscape symbol difference perception analysis method. Addressing the challenges of accurately identifying language boundaries, misaligned zoning labels, and missing cultural background translations when foreign and Chinese texts are mixed on multilingual signs, the method employs an image acquisition and positioning module to capture the real-world view of the sign and bind it to map anchor points. Sign types are categorized and stored in a database based on appearance features. Optical character recognition (OCR) is used to extract character stroke outlines. Combining unit area density and uniformity of side margins, the method determines the increase in character density and the early threshold for language switching. A convolutional neural network is then used to learn features from the density sequence along the scanning direction, outputting a character arrangement fault morphology. Candidate positions exceeding the threshold are selected for language identification, and a character segmentation module divides stroke groups. Density and uniformity of margins are used as constraints to verify and determine language boundaries. A gradient boosting decision tree is then used to fuse the fault morphology and boundary positions to generate boundary markers, completing the zoning language labeling. Furthermore, cultural translations and translation strategy annotations are added to signs with super-linguistic backgrounds. Ultimately, this method achieves a comprehensive effect of accurate language boundary identification, cultural adaptation of translations, and visual annotation of error states. Attached Figure Description

[0016] Figure 1 This is a flowchart of a multi-contextual linguistic landscape symbol difference perception analysis method according to the present invention.

[0017] Figure 2 This is a schematic diagram of a multi-contextual linguistic landscape symbol difference perception analysis method according to the present invention.

[0018] Figure 3This is another schematic diagram of a multi-contextual linguistic landscape symbol difference perception analysis method according to the present invention. Detailed Implementation

[0019] To further understand the content of this invention, a detailed description of the invention is provided in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0020] like Figures 1-3 This embodiment of a multi-context linguistic landscape symbol difference perception analysis method may specifically include: S101. Capture real-world images of multi-context signs using an image acquisition module, record geographic coordinates using a positioning module, and bind anchor points to Google Maps and Gaode Maps. Classify signs based on their appearance characteristics to obtain sign types, and write the sign types into the language landscape integrated database.

[0021] The image acquisition module captures real-scene images of multi-context signs, while the positioning module simultaneously records the latitude and longitude coordinates of the shooting point. These coordinates are then bound to map anchor points on Google Maps and Gaode Maps, resulting in real-scene images of signs with dual map anchor points. Based on these images, the appearance features of the signs—border shape, base material, pasting method, and signboard graphic—are extracted. The signs are then categorized according to these appearance features. The four-dimensional appearance features of the border shape, base material, pasting method, and signboard graphic are used to categorize the signs. Hand-painted or spray-painted marks on walls are classified as graffiti; signs with graphic and textual descriptions are classified as explanatory signs; signs with signs featuring coffee cups, wine glasses, or plates are classified as cafes, bars, and restaurants under the food and beverage category; signs with room signs are classified as accommodations; and signs with display shelves are classified as retails. The sign types and corresponding map anchor points are then associated and written into the language landscape database.

[0022] In one implementation, the image acquisition module uses a camera component integrated into a mobile terminal or inspection equipment to take frontal photos of multi-context signs such as those in urban commercial streets, scenic area entrances, and hotel entrances. The positioning module uses a global navigation satellite receiver unit in conjunction with a mobile network base station to assist in positioning and obtain latitude and longitude coordinates. The latitude and longitude coordinates, along with the shooting timestamp, are added to the metadata field of the real-scene image.

[0023] Specifically, the coordinate node interfaces of Google Maps and Amap are used to bind map anchor points for the latitude and longitude coordinates. An anchor point A1 is placed on the Google Maps layer, and an anchor point A2 is placed on the Amap layer. A1 and A2 share the same set of latitude and longitude values, respectively representing the location on the overseas and domestic base maps. A1 and A2 are associated with the unique identifier of the signboard's real-world image and stored together, forming the signboard's dual map anchor points. This means that a single real-world image of the signboard retains a corresponding location node in each of the two map coordinate systems, resulting in a signboard real-world image with dual map anchor points. This facilitates subsequent location of the signboard across different map engines.

[0024] Preferably, coordinate nodes whose landing point offset exceeds a preset threshold are subject to secondary correction so that the anchor points of the two maps point to the same geographical location.

[0025] It is understood that the dual map anchor points refer to two sets of latitude and longitude coordinates obtained by marking the same sign on satellite maps and street view maps respectively, denoted as P1 and P2, and the landing point offset d is the straight-line distance on the ground between P1 and P2. When the landing point offset d exceeds a preset threshold of 5 meters, the coordinate node is corrected a second time. The preset threshold of 5 meters is determined based on the typical deviation statistical distribution of the dual map coordinate system transformation.

[0026] The correction method is as follows: using identifiable reference features in the real-world image of the signboard as a benchmark, and combining the azimuth angle of the street view shooting, P1 and P2 are weighted and averaged, with the weights allocated according to their respective location confidence levels, to obtain the fused coordinates P. * =w1×P1+w2×P2, where P1 and P2 are latitude and longitude coordinate vectors, w1+w2=1, the side with higher confidence is weighted at 0.7, with a value range of 0.6 to 0.8, based on the dual-map location confidence comparison experiment calibration, and the other side is weighted at 0.3, so that the anchor points of the two maps are ultimately aligned to the same geographical location.

[0027] Understandably, the appearance features of the signboard real-world image with dual map anchor points are extracted. These appearance features encompass four dimensions: the signboard's border shape, substrate material, installation method, and signage. The border shape distinguishes between a regular rectangular border and spray marks without a fixed border; the substrate material distinguishes between supporting media such as acrylic, metal, and walls; the installation method includes hanging, wall embedding, and column fixing; and the signage incorporates visual elements such as coffee cups, wine glasses, plates, room bed maps, and shelf display windows. Based on the aforementioned appearance characteristics, the signage is categorized as follows: if the frame is regular, the base material is acrylic or metal, and it is suspended or fixed to a pillar with printed lettering, it is classified as a printed sign; if the base material is the wall itself, there is no fixed frame, and it shows hand-painted or spray-painted marks, it is classified as graffiti; if the base material is metal or acrylic, it is embedded in the wall and has graphic and textual explanations, it is classified as an information sign; if the storefront includes coffee cup, wine glass, or plate signs, and the base material is mostly acrylic illuminated panels and it is suspended, it is classified as a coffee shop, bar, or restaurant under the food and beverage category; if it includes room signs, the base material is a large metal plate, and it is fixed to a pillar or suspended, it is classified as an accommodation category, such as a hotel or guesthouse; if it includes a shelf display window, the base material is glass or acrylic, and it is embedded in the wall, it is classified as a retail category, such as a grocery store. The language landscape database uses map anchor point numbers as the primary key. Each record uniformly includes: anchor point number, latitude and longitude coordinates, original image path, sign type, morphology field, critical point field, boundary field, language field, translation field, and background information field. The morphology field stores geometric description parameters of the sign's outer contour, including aspect ratio, tilt angle, and corner coordinate sequence, normalized to 0 to 1. The critical point field stores the set of key coordinate points for sign edge segmentation. This database is created as an empty table during system initialization and the corresponding fields are gradually filled in as the system progresses.

[0028] For example, the sign type, along with the latitude and longitude coordinates of the corresponding map anchor point and the original real-scene image path, are written into the language landscape integrated database in the above-mentioned structured record form, providing a carrying location for subsequent language and translation information added at the same anchor point.

[0029] S102. Using optical character recognition technology, extract the outline of character strokes within the signboard area, identify the density of foreign and Chinese characters within a unit area, and the uniformity of the width of the blank space on both sides of the characters. Based on this, determine the increase in character density at each position along the scanning direction of the signboard, identify the position where the density increases and determine it as the early threshold for language switching.

[0030] A real-world image of a signboard with dual map anchor points was retrieved from a comprehensive language landscape database. Optical character recognition (OCR) was used to extract the stroke outlines of each character in the signboard area. Based on the connected components of these stroke outlines on the pixel grid, the bounding box of each character was delineated. A scanning direction was set along the long side of the signboard, and the bounding boxes of the characters along this scanning direction were sorted by position to obtain a character sequence along the scanning direction. For Chinese and foreign characters in the character sequence, the number of bounding boxes per unit area was counted to obtain the density. The width of the blank pixels between adjacent character bounding boxes was measured, and the standard deviation of each blank pixel width was calculated, with the reciprocal taken as the uniformity of the white space width on both sides of the characters. A fixed-length sliding window was used along the scanning direction, and the average density within the window was calculated and multiplied position by position by the uniformity of the white space width to determine the increase in character density at each position along the scanning direction of the signboard. The character density increase result is divided position by position along the scanning direction to obtain the density increment curve between adjacent positions. If the density increment at a certain position exceeds a preset threshold and the uniformity of the white space width on the left side of the position is higher than that on the right side but the density is opposite, then the position is determined to be a density increase jump point. The scanning coordinates of the density increase jump point are marked as the language switching early threshold.

[0031] In one implementation, the comprehensive language landscape database consists of three parts: an image storage area, a morphological field area, and a critical point field area. During database construction, the collected real-world images of signs are first uniformly numbered and entered into the database. Then, metadata fields are added to each record. These metadata fields include four items: latitude and longitude of the shooting point, latitude and longitude of the actual installation point of the sign, sign type, and collection time. The latitude and longitude of the shooting point are directly read by the satellite positioning module of the acquisition terminal. The latitude and longitude of the actual installation point of the sign are calculated by the acquisition terminal at the moment of shooting based on the distance value and azimuth angle returned by the ranging sensor, offsetting the coordinates of the shooting point. These two are collectively referred to as dual map anchor points, used to simultaneously reconstruct the shooting perspective and the sign's position on the map later. The morphological field area is used to backfill the subsequently extracted stroke outline vectors, and the critical point field area is used to backfill the coordinates of the skeleton endpoints and intersections. These two are linked one-to-one with the image storage area through record numbers. The optical character recognition module reads the real-scene images of the signboard one by one in the recording order. It performs perspective correction on the circumscribed rectangular area of ​​the signboard in the image to ensure the signboard plane is parallel to the image plane, thus avoiding character distortion caused by the shooting angle that could interfere with subsequent stroke contour extraction. The optical character recognition module employs a convolutional neural network-based text detection and character segmentation pipeline. First, it binarizes the signboard area, separating the text pixels from the background pixels. Then, it performs a thinning operation on the binary image to obtain the skeleton of the character strokes. The endpoints and intersections of the skeleton together constitute the stroke contour, which is written into the morphology field area and the critical point field area, respectively.

[0032] Specifically, for the stroke outline, eight-neighbor connected components are calculated on the pixel grid. Each connected component delineates the bounding box of a character, which is defined by the smallest bounding rectangle of the connected component. The pixel coordinates of its upper left and lower right corners are recorded. A scanning direction is set along the long side of the signboard, and the bounding boxes of the characters along the scanning direction are sorted according to their upper left horizontal coordinates to obtain a character sequence arranged along the scanning direction.

[0033] It should be noted that the density of character arrangement is calculated using a sliding rectangular window along the long side of the signboard. The window height H is equal to the net height of the signboard, and the window length L is 3 to 5 times the average height h of the character's outer frame within the signboard area. In this embodiment, L = 4 × h is used. The rectangular window is slid from left to right with a step size s = h / 2. The number N of characters that fall completely into the window or whose outer frame center point falls into the window is counted. The density r at this position is defined as the ratio of the number of foreground pixels S1 of the character strokes within the scanning window to the total number of pixels S0 in the window, r = S1 / S0, with a value between 0 and 1. For dense Chinese characters, r is typically 0.55 to 0.75, and for sparse foreign characters, r is typically 0.25 to 0.45, based on statistical calibration of typical multilingual signboard character samples.

[0034] In one embodiment, the measurement process for the uniformity of whitespace width is as follows: For adjacent character bounding boxes along the scanning direction, the horizontal blank pixel width between the right boundary of the preceding bounding box and the left boundary of the following bounding box is measured. The horizontal blank pixel width between adjacent character bounding boxes is also measured. At a certain position, five consecutive blank widths are taken on each side, and their standard deviations are calculated. The reciprocal of the standard deviation is used as the uniformity of whitespace width on that side. The larger the value, the more uniform the character spacing distribution on that side. The larger the value of the uniformity of whitespace width, the more uniform the character spacing distribution on that side; the smaller the value, the more discrete the character spacing distribution on that side. The whitespace between Chinese characters is usually narrower and more uniform. Although the spacing between letters within foreign language words is narrow, the spacing between words is wider, resulting in a lower uniformity of whitespace width on the foreign language side.

[0035] Specifically, a fixed-length sliding window is used along the scanning direction, with the window length being 6 to 10 times the average character width. The arithmetic mean of the character density *r* within the window is calculated to obtain the average density *d* at the center of the window. The uniformity of the whitespace width *u* at the same position is taken, and the product of *d* and *u* is used as the increase in character density at that position, denoted as D_. density =d×u, calculated position by position along the scanning path and arranged in order of position coordinates, forming a line with position as the horizontal axis and D_ densityThe one-dimensional numerical sequence with values ​​on the vertical axis serves as the input for subsequent convolutional neural network and normalization processing. The character density increase results are connected along the scanning direction to form a curve. The dense Chinese area and the sparse foreign language area show varying heights on the curve, and the transition areas between these steps are candidate language boundaries. Further, the character density increase results are differentially analyzed position by position along the scanning direction, i.e., the difference between the character density increase results at adjacent positions is taken to obtain the density increment curve between adjacent positions. The peak value on the density increment curve corresponds to the jump position where the character density changes from sparse to dense or from dense to sparse. To eliminate the dimensional influence caused by differences in character size, the difference result is divided by the average character height h of the real-world image of the signboard to obtain the dimensionless density increment ΔD, where ΔD = (D_ {i+1} -D_ i ) / h, where h is the average height of the character (pixels), and the jump criterion is that the dimensionless density increment ΔD exceeds a preset threshold of 0.4 (ranging from 0.3 to 0.5, based on the sample statistical calibration of the density jump amplitude at the Chinese-foreign language boundary). If a certain position on the density increment curve satisfies ΔD greater than 0.4, and the uniformity of the white space on the left side of the window at that position is greater than 0.2 (absolute difference) on the right side, while the density of the layout is less than 0.2 (absolute difference) on the right side, then that position is determined to be a density increase jump point. Here, "early" means that the jump point is located before the right boundary of the main language segment marked by step S102, that is, the language transition occurs before the end of the main segment of the page, hence the name language switching early threshold. The language landscape comprehensive library is initialized and established in step S101, including substructures such as map anchor point field, morphology field, threshold point field, and language label field. The threshold point field uses the horizontal coordinate x of the scan coordinate and the latitude and longitude of the corresponding map anchor point as index items, which are called by the verification step in step S104.

[0036] S103. Based on the result of increased character density and the early threshold of language switching, a convolutional neural network is used to learn the features of the density-increasing sequence along the scanning direction, and output the character arrangement tomographic morphology at each position.

[0037] The language switching early threshold and the corresponding character density increase result are read from the critical point field of the language landscape comprehensive database. The scan direction is extended to the left and right sides by a preset length with the language switching early threshold as the center. The local numerical sequence of the character density increase result is extracted to obtain the density increase sequence. The density increase sequence is normalized to the maximum and minimum values ​​in terms of numerical amplitude so that the sequence amplitude falls within the range of 0 to 1. Then, the normalized density increase sequence is segmented and labeled with three segments: left, transition, and right, with the language switching early threshold as the marker. The sequence length is then padded with zero values ​​to a fixed length to obtain the density increase sequence with segmentation markers. A convolutional neural network is used to learn features from the density-increasing sequence with segmented labels. The convolutional neural network consists of a one-dimensional convolutional layer, a max-pooling layer, and a fully connected layer. The one-dimensional convolutional layer slides the convolutional kernel along the scanning direction to extract features from the local part of the sequence, obtaining the convolutional response in multiple channels. The max-pooling layer downsamples the convolutional response and retains the local abrupt response. The fully connected layer performs a position-wise weighted mapping on the downsampled feature vector, outputting a fault morphology vector containing three components: density increase magnitude, density decrease magnitude, and fault straightness. The fault morphology vector is categorized position by position along the scanning direction. If the difference between the density increase and density decrease in the fault morphology vector exceeds a preset threshold and the fault straightness is lower than a preset threshold, the position is determined to be a steep fault morphology. If the difference is lower than the preset threshold and the fault straightness is higher than the preset threshold, the position is determined to be a gradual fault morphology. The remaining positions are classified as transitional fault morphologies. The fault morphology vectors at each position are integrated with the corresponding morphology labels according to the scanning direction coordinates to obtain the character-arranged fault morphology labeled position by position along the scanning direction.

[0038] The character arrangement fault morphology serves as a crucial basis for subsequent language boundary determination. Essentially, it represents the geometric shape of density changes in a density-increasing sequence near the early language switching threshold. The transition between the dense arrangement of Chinese square characters and the sparse arrangement of foreign long characters leaves step-like undulations on the density-increasing sequence. The steepness of these undulations, the difference in platform height, and the edge symmetry constitute the fault morphology. The early language switching threshold is determined by statistically analyzing the stroke percentage of characters column by column along the scanning direction in the preceding steps. The first difference of the percentage curve is calculated, and the horizontal coordinate position exceeding the empirical threshold of 0.35 is taken as the warning point where the density begins to rise before language switching. The character density increase result is the cumulative count sequence of stroke pixels corresponding to each horizontal coordinate under the same scanning direction. A larger value indicates a higher character density in that column, stored in a one-dimensional array in horizontal coordinate order. The comprehensive language landscape database is constructed during the preceding image acquisition and preprocessing stages. Using map anchor point numbers as the primary key, it includes fields for real-scene images, critical points, morphology, and semantics. The critical point field stores the x-coordinate of the language switching threshold for each signboard image and the corresponding numerical values ​​of character density increase. During retrieval, the entry is located by map anchor point number, and the language switching threshold and corresponding character density increase result are retrieved. The average character width is obtained from the preceding character segmentation results, specifically by averaging the bounding box widths of all segmented characters in the image, denoted as w.

[0039] Specifically, taking the language switching early threshold as the center, a preset length L is extended to the left and right along the scanning direction. Local numerical sequences falling within this range are extracted from the character density increase results to obtain a density increase sequence. The density increase sequence is then subjected to maximum-minimum normalization, i.e., the maximum and minimum values ​​in the sequence are taken, and each value is subtracted from the minimum value and divided by the difference between the maximum and minimum values, ensuring the sequence amplitude falls within the 0-1 range to eliminate the influence of shooting illumination differences on the amplitude. Further, using the language switching early threshold as a marker, the normalized density increase sequence is divided into three segments along the scanning direction: left, transition, and right. Samples within a certain window width to the left of the threshold are assigned to the left segment, samples within a certain window width near the threshold are assigned to the transition segment, and samples within a certain window width to the right of the threshold are assigned to the right segment. For density increase sequences where the interception range exceeds the limit due to the threshold being close to the image edge, resulting in an actual length less than 2×L, zero values ​​are added to both ends to make the length of each sample uniform, resulting in a density increase sequence with segmentation markers. A convolutional neural network is used here to learn the fault geometry from the density-increasing sequence with segmented labels. Its structure consists of a one-dimensional convolutional layer, a max-pooling layer, and a fully connected layer. The one-dimensional convolutional layer is responsible for local shape extraction, the max-pooling layer is responsible for feature compression, and the fully connected layer is responsible for mapping the shape to a morphological vector.

[0040] It should be noted that the one-dimensional convolutional layer is configured with multiple convolutional kernels of different sizes, with kernel widths of 3, 5, and 7 respectively. Each convolutional kernel slides along the scanning direction and performs a dot product with the local segment of the density-increasing sequence it covers to obtain the response value. During the training phase, the weights inside the convolutional kernels are gradually adjusted through backpropagation using density-increasing sequence samples with morphological labels as input and pre-annotated morphological labels as supervision signals. After multi-channel parallel output, a multi-channel convolutional response is formed, with different channels corresponding to local shape responses such as "rising edge," "falling edge," "flat top," and "valley."

[0041] Specifically, the max pooling layer slides along the scanning direction with a fixed window length to the multi-channel convolutional response, taking the maximum response value within the window as the output. The fixed window length is 3, the sliding step size is 2, and there is an overlap of 1 sampling point between adjacent windows. The length of the output sequence is approximately 1 / 2 of the input length, thereby achieving double downsampling. The fully connected layer performs weighted summation and bias on the downsampled feature vector, and outputs three values ​​after the activation function: the density increase magnitude, the density decrease magnitude, and the fault straightness, which together constitute the fault morphology vector.

[0042] Understandably, the density increase reflects the vertical span of the density-increasing sequence to the left of the critical point, climbing from the baseline to a higher level; the density decrease reflects the vertical span of the density-increasing sequence to the right of the critical point, falling back to the baseline from a higher level. Fault straightness is characterized by the product of two dimensionless ratios. The language switching advance threshold is determined according to a unified calibration method and written into the critical point field; this step directly reads it. The fault straightness P is calculated as follows: First, calculate the length L and variance V of the top platform to obtain P0 = L / (V + 0.001). Then, map P = P0 / (P0 + 1) to the interval between 0 and 1, thus comparing it with the rise and fall amplitudes on the same scale. For example, at a certain position, when L is 6 and V is 0.5, P0 is 12, and after normalization, P is approximately 0.92, which is judged as high flatness. Here, 0.001 is a zero-division protection to prevent division when the variance is 0, and is an empirically selected value. The three components correspond to the bulge direction, bulge symmetry, and platform stability at the boundary between the dense Chinese area and the sparse foreign language area, respectively.

[0043] In one embodiment, the fault morphology vector is categorized position by position along the scanning direction. The threshold for the difference between the density increase and decrease is set to 0.2, and the threshold for fault straightness is set to 0.4. Both the difference threshold and the fault straightness threshold are determined based on the statistical boundary of manually annotated samples of abrupt and gradual fault morphologies. If the difference is less than 0.2 and P is greater than 0.4, the position is determined to be a gradual fault morphology, corresponding to a soft transition where the spacing between characters gradually increases. The remaining positions are classified as transitional fault morphologies, corresponding to a complex situation where the transition segment contains mixed punctuation marks or graphic symbols.

[0044] For example, on a commercial street, at the intersection of the Chinese sign "Old Beijing Fried Noodles" and the foreign sign "Beijing NoodleHouse," the density increase in the fault morphology vector is higher than the decrease, and the flatness is low, which is identified as a steep fault morphology. Conversely, at the intersection of the sign "Coffee" and the sign "Coffee," the spacing between the characters gradually changes, and the flatness is high, which is identified as a gradual fault morphology. The language landscape database uses the collection point as the primary key and includes four types of fields: a map anchor point field recording latitude, longitude, and orientation; a language tag field recording Chinese and foreign language segments; a critical point field recording early language switching thresholds; and a morphology field recording the fault morphology sequence generated in this step. In this embodiment, the fault morphology vectors at each location and their corresponding morphology tags are integrated into an ordered sequence according to the scan direction coordinates, i.e., the character arrangement fault morphology, written back to the morphology field, maintaining a reference relationship with the corresponding map anchor point and the early language switching threshold.

[0045] S104. Select candidate positions that exceed the language switching threshold from the character arrangement fault pattern, classify the candidate positions according to language, obtain the boundary region recognition result, process the boundary region recognition result through the character segmentation module, segment the stroke groups of foreign language and Chinese, and use the arrangement density and the uniformity of the white space width of the characters on both sides as constraints to verify the segmented stroke groups and determine the language boundary position.

[0046] Extract the morphological label sequence and corresponding coordinate sequence along the scanning direction from the character arrangement tomographic morphology. Compare the coordinates of each position in the morphological label sequence with the abscissa of the language switching early threshold. Filter out the position coordinates that are located within a preset range near the language switching early threshold and are accompanied by abrupt or gradual tomographic morphology to obtain a candidate position set. For each candidate position in the candidate position set, extract the character stroke outlines around it and send them to a random forest classifier. Determine the language of the characters to the left and right of the candidate position based on the stroke direction, stroke thickness ratio, and character height-to-width ratio to obtain the boundary region recognition result. Based on the boundary region identification results, the connected components corresponding to the character stroke contours within the boundary region are reclassified along the scanning direction using the character segmentation module. The height and thickness of the stroke contours are measured character by character from the candidate position to the left and right. Using the average value of adjacent characters as a benchmark, a segmentation line is inserted where the deviation in height or stroke thickness exceeds a preset difference. The strokes on both sides of the segmentation line are merged according to their respective languages ​​to obtain two sets of segmentation results: foreign language stroke groups and Chinese stroke groups. The horizontal coordinate boundaries of the foreign language stroke groups and the Chinese stroke groups are recorded. The horizontal coordinate boundary between the foreign language stroke group and the Chinese stroke group is verified by the density of the stroke arrangement and the uniformity of the white space width on both sides. The density of the stroke arrangement and the uniformity of the white space width are measured at the junction of the foreign language stroke group and the Chinese stroke group. If the difference in density on both sides of the junction exceeds a preset threshold and the uniformity of the white space width shows an opposite distribution of high and low, then the dividing line at the junction is determined to be accurate. The horizontal coordinate of the dividing line is determined as the language boundary position. If any constraint condition is not met, the character segmentation module re-merges the stroke groups once according to the deviation and outputs the result.

[0047] Precisely pinpointing the language boundary is a prerequisite for the execution of multi-context sign language labeling. The language switching advance warning threshold x0 is determined using a unified calibration method and stored in the critical point field. This step directly reads the threshold by key name; its physical meaning is the estimated language switching advance warning point, stored as a single field in the language landscape comprehensive database for subsequent steps to directly access via key name. Although the character arrangement tomographic morphology provides a sequence of morphological labels and corresponding coordinates along the scanning direction, the morphological labels only indicate the geometric shape of density changes. The true language boundary requires four processing steps: candidate screening, language identification, stroke segmentation, and multi-constraint verification. In one implementation, the candidate position screening process uses the x-coordinate of each coordinate point in the morphological label sequence... i The difference between x0 and d is d = |x0|. i -x0|, the coordinates of positions that fall within the preset range and exhibit abrupt or gradual fault patterns are included in the candidate position set; transitional fault patterns are not included in the candidate set due to their smaller density span. The preset range is 5 to 8 times the average width of the character, which covers reasonable offsets near the critical point while avoiding the introduction of stroke jumps unrelated to the boundary.

[0048] Specifically, the Random Forest classifier is used to determine the language of the characters to the left and right of each candidate position. During the training phase, the Random Forest classifier uses manually labeled signboard character fragments as samples. The sample features include three sets of indicators: stroke direction histogram, stroke thickness ratio, and character height-to-width ratio. The stroke direction histogram statistically analyzes the pixel proportion of character strokes along four directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Chinese characters are relatively evenly distributed in these four directions, while foreign characters are concentrated at 0 degrees and 90 degrees. The stroke thickness ratio is the quotient of the width of the thickest and thinnest pixel of a stroke; Chinese characters tend to have a larger stroke thickness ratio, while foreign characters tend to have a smaller one. The character height-to-width ratio is the quotient of the height and width of the character's bounding box; Chinese characters have a height-to-width ratio close to 1, while foreign characters typically have a height-to-width ratio greater than 1.5. Multiple decision trees each give a language discrimination vote, and the language of the characters to the left and right of the candidate position is given by majority vote. After merging the left and right language labels of all candidate positions, the boundary region recognition result is obtained.

[0049] It should be noted that the boundary region identification results are recorded in the form of triplets, including the horizontal coordinate of the candidate position, the language label on the left, and the language label on the right, for direct use by the downstream character segmentation module. Further, the character segmentation module uses the candidate position as an anchor point to reclassify the connected components corresponding to the character stroke outlines within the boundary region along the scanning direction. Connected components are originally based on independent characters. At the boundary between Chinese and foreign languages, some foreign letter connecting strokes and Chinese radicals may be mistakenly grouped into the same connected component due to pixel adjacency. Therefore, it is necessary to reclassify them according to the gaps along the scanning direction. Specifically, starting from the candidate position, the height and thickness of the stroke outline are measured character by character, moving left and right. The height is the pixel span between the upper and lower edges of the connected component, and the stroke thickness is the median width of the stroke skeleton within the connected component. Using the average height and average stroke thickness of several adjacent characters on one side of the candidate position as a benchmark, if the deviation of a character's height or stroke thickness from the benchmark exceeds a preset difference, a segmentation line along the vertical direction of the signboard is inserted before that character. The preset difference is 30% of the baseline value. Specifically, the baseline value ranges from 25% to 35%, determined statistically based on the upper limit of natural fluctuations in character height and stroke thickness within the same language. The density difference threshold is set to 0.3. This density difference threshold is determined statistically based on samples showing relative density differences at the intersection of Chinese and foreign languages. Uniformity in reverse distribution is judged by one side being 1.5 times greater than the other.

[0050] In one embodiment, the strokes on both sides of the dividing line are assigned to corresponding sets based on the left and right language labels in the boundary region identification results. The strokes before the left dividing line are assigned to either the foreign language stroke group or the Chinese stroke group, and the strokes after the right dividing line are assigned to the other stroke group. The horizontal coordinate boundaries of the foreign language stroke group and the Chinese stroke group are recorded, that is, the left and right pixel column numbers at the junction of the two stroke groups.

[0051] Understandably, the horizontal coordinate boundary obtained solely from stroke shape segmentation may still be offset by cursive script, artistic fonts, and decorative borders. Therefore, two constraints—layout density and uniformity of white space width on both sides—are introduced for verification. The verification process involves taking an observation window at the junction of the foreign language stroke group and the Chinese stroke group, centered on the horizontal coordinate boundary, extending to both sides. The width of each observation window is three times the average width of the characters. Within each observation window, the layout density and uniformity of white space width are measured. The density of stroke arrangement is denoted as D, which is defined as the ratio of the total number of foreground pixels S in the observation window to the area A of the observation window, and then multiplied by the normalization factor of the sum of the average width w of the outer bounding box of the character in the window and the average character spacing g, i.e., D=(S / A)·[w / (w+g)], where S is the number of foreground pixels in the observation window, A is the area of ​​the observation window, w is the average width of the outer bounding box of the character in the window, g is the average character spacing, and w / (w+g) is the character proportion normalization factor, which normalizes D to between 0 and 1. The larger the value, the more compact the stroke arrangement. In specific implementation, the image in the observation window is first binarized to obtain the foreground mask, the number of foreground pixels S is counted, and then the outer bounding box of each character is obtained through connected component analysis. The average width w is taken, and the average distance from the right boundary of the adjacent outer bounding box to the left boundary of the next outer bounding box is taken as g. Substituting into the above formula, D is obtained. D1 and D2 are obtained from the observation windows on both sides. When the absolute value of the difference between D1 and D2 is greater than 0.3, it is determined that the density drop at the junction is abnormal and the horizontal axis boundary needs to be backtracked and corrected.

[0052] For example, if the density difference between the two sides of the junction exceeds a preset threshold, and the uniformity of the white space width shows a reverse distribution with one high and one low (i.e., the white space on the Chinese side is narrow and uniform, while the white space on the foreign language side is wide and discrete), then the dividing line at the junction is determined to be accurate, and the abscissa of the dividing line is determined as the language boundary position. Preferably, if either of the two constraints is not satisfied, the character segmentation module uses the character on the side with the larger deviation as a reference, shifts it by one character width in the direction of decreasing deviation, re-merges the stroke groups, verifies again, and directly outputs the abscissa of the dividing line obtained in this verification as the language boundary position, thereby avoiding the hidden danger of non-convergence of the verification loop. The language boundary position is written back to the language landscape integrated database along with the map anchor points, leaving an input interface for the generation of boundary markers between the foreign language and Chinese regions.

[0053] S105. Using a gradient boosting decision tree, the character arrangement fault morphology and language boundary position are integrated to generate the boundary marker between the foreign language area and the Chinese area.

[0054] The character arrangement fault morphology is read from the morphology field of the language landscape comprehensive database, and the language boundary position is read from the critical point field. Using the horizontal coordinate of the language boundary position as the anchor point, three components—density increase, density decrease, and fault straightness—are extracted from the character arrangement fault morphology within a preset length on both sides of the anchor point, along with the corresponding morphological label. These components are then concatenated with the horizontal coordinate of the language boundary position in the order along the direction of the signboard to obtain a fused feature vector. A gradient boosting decision tree is used to perform fusion learning on the fused feature vector. The gradient boosting decision tree is built one decision tree at a time in a multi-round iterative manner. Each decision tree uses the residual of the previous round as the fitting target to split and distinguish the components of the fused feature vector. The leaf nodes of the decision tree output the language attribution score of the anchor point position. After weighted summation by multiple decision trees, positive values ​​represent the Chinese region attribution and negative values ​​represent the foreign language region attribution, resulting in a sequence of language attribution scores for each anchor point along the direction of the signboard. The language attribution score sequence is segmented and aggregated along the direction of the signboard. If the language attribution scores of adjacent positions have opposite signs and the difference crosses a preset zero threshold, then the position is determined to be the boundary point between the foreign language area and the Chinese area. The horizontal coordinate of each boundary point is recorded as the endpoint of the segment. The interval between adjacent boundary points is labeled as the foreign language area or the Chinese area according to the mean sign of the language attribution scores within the interval. The corresponding foreign language area label or Chinese area label is embedded at the endpoint of the segment to obtain the boundary marker between the foreign language area and the Chinese area.

[0055] The boundary marking between the foreign language and Chinese language sections is based on the already determined language boundary locations. The entire signboard is divided into several sections with clearly defined language affiliations along its direction, and labels are added between these sections for direct use in subsequent annotation and translation processes. Previous processes only provided the boundary coordinates at individual points. However, when a signboard actually contains multiple sections interspersed with both Chinese and foreign language text, multiple points need to be linked together to form section boundaries. Therefore, a gradient boosting decision tree is introduced to jointly fuse the fault morphology and boundary locations. In one implementation, the character arrangement fault morphology is read from the morphology field, and the language boundary location is read from the critical point field. The two are matched one-to-one according to map anchor point numbers and aligned by array subscripts, ensuring that the morphological sequence and boundary coordinates under the same real-world image of the signboard can be retrieved synchronously.

[0056] Specifically, using the horizontal coordinate of the language boundary location as the anchor point, a neighborhood of a predetermined length is selected to the left and right along the direction of the signboard. From the character arrangement fault pattern, three components are extracted: density increase rate, density decrease rate, and fault flatness, along with corresponding morphological labels for each position within the neighborhood. These three components represent the vertical span and platform stability of character density within the neighborhood, respectively, while the morphological labels indicate three types: abrupt change, gradual change, and transition. The extracted components, morphological labels, and the horizontal coordinate of the language boundary location are concatenated in order along the direction of the signboard to obtain a fused feature vector. The fused feature vector carries both local geometric information and global coordinate information, avoiding discrimination bias caused by a single dimension. Gradient boosting decision trees serve as the carrier of fusion learning. Their principle is to build several shallow decision trees through multiple iterations. Each subsequent decision tree uses the negative gradient of the current accumulated output under logarithmic loss as the fitting target, gradually approximating the true language attribution discrimination function. During the training phase, pre-labeled language-attributed fusion feature vector samples are used as input. The four continuous components of the fusion feature vector—density increase / decrease, fault straightness, and boundary coordinates—are binary searched at split nodes according to candidate thresholds, assigning samples to left and right subsets. The three morphological labels—abrupt, gradual, and transitional—are first expanded using one-hot expansion, resulting in three indicator components with values ​​of 0 or 1. These are then binary-sorted and merged at split nodes according to category subsets; for example, abrupt changes are assigned to the left subset, and gradual and transitional changes to the right subset. For both continuous and discrete indicator components, the decrease in the sum of squared negative gradient residuals before and after splitting is used as the gain metric. The component with the largest gain and its split point are selected as the splitting criteria for that node, until a preset tree depth of 6 layers is reached or the single gain falls below 1e-4, at which point growth stops. Specifically, during the online operation phase, the gradient boosting decision tree receives the fusion feature vector as anchor-point input, and the leaf nodes of the decision tree directly output the language attribution score at that anchor point. Multiple decision trees each give a score, and the scores are weighted and accumulated after pre-training, and then the sign is determined: positive values ​​represent the Chinese region and negative values ​​represent the foreign language region. The magnitude of the value reflects the strength of the region.

[0057] Understandably, the language attribution scores of all anchor points are arranged sequentially along the horizontal axis of the signboard, thus obtaining the language attribution score sequence. This sequence shows continuous positive values ​​in the Chinese-dense sections, continuous negative values ​​in the foreign-dense sections, and sign reversal at the boundaries.

[0058] In one embodiment, the language attribution score sequence is segmented and aggregated along the signboard, position by position. The language attribution score symbols of adjacent positions are compared. If the language attribution score symbols of adjacent positions are opposite and the absolute value of the difference crosses a preset zero-point threshold of 0.2, where the preset zero-point threshold is calibrated based on sample experiments that filter out symbol jitter and retain genuine cross-regional transitions, then that position is determined to be a language switching boundary point. This threshold filters out slight symbol jitter while retaining genuine cross-regional transitions. The x-coordinates of all boundary points are then compared with the starting x-coordinate x0 at the left end and the ending x-coordinate x at the right end of the signboard. n These are all incorporated into the segment endpoint set, which consists of two sources: one is the language boundary points located inside the signboard, detected by zero-crossing symbols; the other is the naturally existing boundary coordinates at both ends of the signboard, with the former being a subset of the latter. After sorting by ascending horizontal coordinates, the endpoint sequence x0, x1, x2…x is obtained. n , with the endpoints x of adjacent segments i With x i+1 Using the intervals between them as units, the mean of the language affiliation score sequence is calculated within each interval. If the mean is positive, the interval is labeled as the Chinese region; if the mean is negative, it is labeled as the foreign language region. The corresponding foreign language region label or Chinese region label is embedded at the endpoints of the intervals on both sides.

[0059] For example, a sign on a commercial street that reads "Old Beijing Fried Noodles BeijingNoodleHouse Welcome" is divided into three sections along its direction. The first section is labeled as the Chinese section, the middle section as the foreign language section, and the last section as the Chinese section again. Chinese section labels, foreign language section labels, and Chinese section labels are embedded at the three section endpoints, respectively, forming the boundary markers between the foreign language and Chinese sections. These boundary markers, along with map anchor points, are written back to the boundary and anchor point fields of the language landscape database according to the above field structure, for use in downstream language labeling processes by primary key retrieval.

[0060] S106. Based on the boundary markers between the foreign language area and the Chinese language area, perform language labeling for the multilingual signs and output the language recognition results of the signs.

[0061] The boundary markers between the foreign language and Chinese regions are read from the boundary field of the language landscape database. Based on the horizontal coordinates of the segment endpoints in the boundary markers, the real-world image of the signboard is divided into several segments along the signboard's direction. The start and end ranges of the horizontal coordinates for each segment and the corresponding foreign language or Chinese region label are obtained. The character stroke outlines within each segment are labeled with the corresponding language based on the foreign language or Chinese region label. The characters in the foreign language region are labeled with the foreign language matched by the character stroke outlines obtained from optical character recognition. The characters in the Chinese region are labeled as Chinese. The language labeling results of all segments are merged to obtain the language identification result of the signboard, which is then written into the language field of the language landscape database.

[0062] The language landscape database is a structured data carrier generated by the preceding sign detection and layout analysis process. It is stored one by one according to the sign instance. Each record contains substructures such as image path field, sign outline field, boundary field, shape field, critical point field and text line field. The boundary field is used to record the boundary information between the foreign language area and the Chinese area, the shape field is used to record the horizontal or vertical orientation type of the sign, and the critical point field is used to record the horizontal coordinate position of the character jump within the text line.

[0063] In one implementation, boundary markers are written into the boundary field of the language landscape database during the preceding process. These markers are organized as a list of segment endpoint coordinates and corresponding foreign language or Chinese language area labels for each segment. For example, the boundary field of a sign is recorded as endpoint lists 120 and 340. Combined with the label sequence "Foreign Language Area," "Chinese Language Area," and "Foreign Language Area," this indicates that coordinates 0 to 120 represent the foreign language area, 120 to 340 represent the Chinese language area, and 340 to the sign's endpoint represent the foreign language area. After reading the boundary markers for the foreign language and Chinese language areas, the sign's real-world image is divided along its direction based on the segment endpoint coordinates in the boundary markers. The segment between the sign's starting point and the first segment endpoint is the first segment, the segment between two adjacent segment endpoints is the middle segment, and the segment between the last segment endpoint and the sign's endpoint is the tail segment. Each segment includes the start and end range of its horizontal coordinates and the corresponding foreign language or Chinese language area label.

[0064] Specifically, the character stroke outlines within each segment are labeled with the foreign language or Chinese language tags. For segments labeled as Chinese, the characters within the segment are uniformly labeled as Chinese; for segments labeled as foreign language, the character stroke outlines obtained from the optical character recognition process in the preceding steps are used as the basis to match letter shapes, ligature rules, and diacritics to determine the specific foreign language. For example, strokes containing Latin letters and accent marks are matched as French or Spanish, those containing Cyrillic letters are matched as Russian, those containing Hangul characters are matched as Korean, those containing kana symbols are matched as Japanese, those containing ligature Arabic letters are matched as Arabic, and Latin letter combinations without the above features are matched as English. The matched foreign language is then used as the language tag for that segment.

[0065] For example, a commercial street sign "Old Beijing Fried Noodles (BeijingNoodleHouse)" is segmented. The first segment (Chinese characters) is labeled as Chinese, and the last segment (foreign characters) is labeled as English because it only contains Latin letters and has no special diacritics. The language labeling results of all segments are merged to obtain the language identification result of the sign. The language landscape database organizes records using map anchor points as the primary key. Each record contains seven substructures: sign identification number, map anchor point coordinates, morphology field, critical point field, language field, original image path, and acquisition timestamp. The morphology field stores the coordinates of the four vertices and tilt angle of the sign's bounding rectangle. The critical point field stores the sequence of abscissas of the segment endpoints obtained from the segmentation. The language field stores an ordered list of binary pairs consisting of the abscissas of the segment endpoints and the language label, for example, written in the format of 0, Chinese, 180, English, 360. The language identification result is written to the language field in the above format and maintains a reference relationship with the morphology field and critical point field through the map anchor point primary key, which facilitates subsequent retrieval of the multidimensional attributes of the same sign by coordinate range.

[0066] S107. Based on the identification results of sign type and language, determine the sign keywords that directly convey the function of the place corresponding to the sign type, generate preliminary translations for each language for tourists of different languages, and write them into the translation field of the language landscape integrated database.

[0067] The system retrieves the sign type and language identification results from a comprehensive language landscape database. Based on the sign type, it searches a pre-established site function keyword database. This database uses the sign type as the key and Chinese sign keywords directly conveying the site function as the value to obtain the corresponding sign keywords. For tourists speaking Chinese, English, Russian, Korean, Japanese, French, Spanish, and Arabic, neural machine translation is used to translate the sign keywords into multiple languages, with Chinese as the source language. Preliminary translations in each language are obtained, and the language identification results are appended to the translation record as the original sign language attribute. These preliminary translations are then written into the translation field of the comprehensive language landscape database in language order.

[0068] In one embodiment, the linguistic landscape comprehensive database is pre-established on a local storage medium, and is divided into three parts: a main sign table partition, an auxiliary vocabulary table partition and a log partition. The main sign table takes the sign number as the primary key, and includes an image path field, a sign type field, a language recognition result field, a shape field, a critical point field, a keyword field and a function label field, wherein the shape field is configured to store geometric description parameters of the sign contour, and the critical point field is configured to store the inflection point coordinate sequence of the sign edge; the auxiliary vocabulary table partition is provided with a place function keyword dictionary, a stop word list and a mapping rule table; the log partition is configured to record timestamps and status codes of each read and write operation. The place function keyword dictionary is established in the auxiliary vocabulary table partition in the form of table entries, each entry takes the sign type as a key and takes Chinese sign keywords that directly convey the place function as a value. Printed signs correspond to name-type keywords, graffiti correspond to content prompt-type keywords, and description boards correspond to interpretation-type keywords; for place categories, cafes under the catering category are mapped to "咖啡馆" (coffee house), pubs are mapped to "酒馆" (pub), restaurants are mapped to "餐厅" (restaurant), hotels under the accommodation category are mapped to "酒店" (hotel), and grocery stores under the retail category are mapped to "杂货店" (grocery store). After reading the language recognition result of the sign and the corresponding sign type, the place function keyword dictionary is retrieved with the sign type as the key to obtain the sign keyword corresponding to the sign type, and the result is written back to the keyword field of the main sign table.

[0069] Specifically, the neural machine translation adopts a translation model based on a Transformer encoder-decoder structure, which takes Chinese as the source language, configures an independent decoder for each of eight target languages and shares the Chinese-side encoder, and generates corresponding preliminary translations for each target language sequentially through encoding and decoding for the sign keywords. With Chinese as the source language and eight languages including Chinese, English, Russian, Korean, Japanese, French, Spanish and Arabic as target languages, an independent Transformer decoder is trained for each target language, and parameters of the same Chinese-side encoder are shared.

[0070] For example, when the sign type is a cafe in the catering category, the sign keyword is "咖啡馆"; after neural machine translation, the keyword is retained as "咖啡馆" in Chinese, translated as "CoffeeHouse" in English, translated as the corresponding word in Cyrillic alphabet in Russian, translated as the corresponding word in Hangul in Korean, translated as "喫茶店" in Japanese, translated as "Café" in French, translated as "Cafetería" in Spanish, and translated as the corresponding word in Arabic alphabet in Arabic; the language recognition result is added to the translation record as the original sign language attribute, and the preliminary translations of each language are written into the translation field of the linguistic landscape comprehensive database together with the map anchor reference according to the language order.

[0071] S108. Manually review and semantically control the preliminary translations, supplement the background cultural translations for signs and scenic spots with extralinguistic backgrounds, and mark the translation strategy type for each translation, taking the coexistence of transliteration and free translation as an example, record the cultural adaptation path behind the Chinese and foreign translations, and form structured cultural annotation data.

[0072] Preliminary translations in each language are retrieved from the translation field of the comprehensive language landscape database. A review process is then conducted to semantically control these preliminary translations. A pre-established cultural term index is used to compare the preliminary translations with each language, identifying the super-linguistic background of signs and scenic spots that carry cultural connotations beyond the literal meaning. Background cultural translations are then added to the identified signs and scenic spots, resulting in a multilingual translation set with cultural annotations. Each translation in this multilingual translation set is labeled with a translation strategy type, including transliteration-preservation, semantic transformation, and hybrid types. The translations present three segments—original name form, transformation method, and target language—recording the cultural adaptation path behind the Chinese and foreign translations. These structured entries, consisting of translation entries, strategy tags, and path descriptions, are then stored in the database, forming structured cultural annotation data.

[0073] In one implementation, the cultural term index is pre-established in the auxiliary terminology section of the linguistic landscape database. It uses the Chinese signboard or scenic spot name as the key and the description of its cultural connotations beyond the literal meaning as the value, covering entries such as Tiananmen Square, candied hawthorn, Wuhou Temple, Goubuli (a type of steamed bun), Coca-Cola, and Huangshan Mountain. The review process reads the preliminary translations in each language from the translation field of the linguistic landscape database, and searches the cultural term index entry one by one, using the original signboard name as a clue. If a match is found, it is determined to be a signboard or scenic spot name with a super-linguistic background. The signboard and scenic spot names are then supplemented with target language background cultural translations corresponding to the original cultural connotations, and merged into the original translated entries, resulting in a multilingual translation set with cultural translations.

[0074] Specifically, the labeling rules for translation strategy types are determined based on the correspondence between the translated text and the original name: those that only retain the pronunciation of the original name and do not convey the literal meaning are classified as transliteration retention type, with a typical example being "Goubuli" which retains the brand pronunciation in the foreign language translation but discards the literal meaning; those that discard the pronunciation of the original name and re-express the meaning in the target language are classified as semantic transformation type, with a typical example being "Tanghulu" which is expressed by translations such as sugar-coated hawthorn skewers; and those that take into account both similar pronunciation and semantic fit are classified as hybrid type, with typical examples being "Coca-Cola" which takes into account both pronunciation and meaning, and "Huangshan" which combines Chinese semantic translation and pinyin.

[0075] Understandably, the cultural adaptation path is recorded in three interconnected segments: the original name form, the conversion method, and the target language. The original name form records the literal structure of the Chinese signboard or scenic spot name, the conversion method records the transliteration retention, semantic transformation, or hybrid method used, and the target language form records the final translated form.

[0076] For example, the original name of the Wuhou Temple entry is a three-character proper name, the conversion method is labeled as hybrid, and the target language is presented as an English translation containing the semantic explanation of "Temple". This translation is written into the entry along with the translation strategy type. The structured entry, which is organized into three columns of translation entry, strategy label, and path description, is entered into the database to obtain the structured cultural annotation data, which maintains a reference relationship with the corresponding map anchor point.

[0077] S109. Based on the structured cultural annotation data, the extracted sign text is subjected to spelling verification, grammar rule scanning, and semantic consistency comparison between Chinese and foreign translations. The error type, error location, original text, and suggested correction text are written into the background information field of the image at the coordinate point. The language error status is marked on the map anchor point. The function of community users to confirm, correct, add, and comment on the sign information is enabled. It also supports one-click jump to Google Maps and Gaode Maps to view the visual route.

[0078] The structured cultural annotation data is read from the comprehensive language landscape database. The extracted signboard text undergoes spell verification using dictionary comparison and grammatical rule scanning using a grammatical rule base. Semantic consistency is compared between the Chinese and foreign language translations in the structured cultural annotation data to obtain the error type, error location, original text, and suggested correction text. These information are then written into the background information field of the image at that coordinate point. Based on the error entries in the background information field, the language error status is marked on the map anchor point at that coordinate point. Community users are then given the ability to confirm, correct, add, and comment on the signboard information. Correction suggestions submitted by community users are written back to the background information field. A one-click redirection to Google Maps and Gaode Maps for viewing visual routes is embedded at the map anchor point.

[0079] In one implementation, after reading the structured cultural annotation data from the language landscape database, three verifications are performed sequentially on the extracted signboard text: spelling verification uses the target language dictionary corresponding to the language identification result as the comparison source, and queries the signboard text word by word. If the dictionary is not matched and there is no transliteration-preserving tag, it is determined to be a spelling error; grammar rule scanning matches with a pre-established target language grammar rule database to identify violations such as mismatched singular / plural nouns, missing articles, and word order inversion; semantic consistency comparison of Chinese and foreign translations uses the target language translation in the structured cultural annotation data as a benchmark and compares it sentence by sentence with the existing translation on the real-world image of the signboard. If the difference exceeds a preset semantic distance threshold, it is determined to be a semantic deviation.

[0080] Specifically, the error information output from the three verification items is organized in the form of a four-tuple, including the error type, the area where the error occurs, the original text, and the suggested correction text. The error type takes one of three values: spelling error, grammatical violation, or semantic deviation; the area where the error occurs is given as a pixel coordinate box on the real-world image of the signboard; the original text records the erroneous content on the signboard; and the suggested correction text is given based on dictionary hits, grammatical rule corrections, or the target language translation from the structured cultural annotation data. All four-tuples are written into the background information field of the image at that coordinate point.

[0081] Understandably, based on the error entries in the background information field, a language error status icon is displayed on the map anchor point at that coordinate, differentiated by error type. The system allows community users to confirm, correct, add, and comment on signage information. When submitting correction suggestions, community users must provide a screenshot of the original text, the corrected text, and an error type label. The system first calls the large language model to re-check the suggestions. If the model confidence score is not lower than 0.85, it passes directly; scores between 0.6 and 0.85 enter the manual review queue, where platform reviewers conduct a second confirmation using the original street view image. Suggestions with a confidence score below 0.6 are directly rejected. If the same entry receives consistent corrections from three or more independent users without objections, the model re-check can be bypassed, and the final manual review can proceed directly. Approved corrections are written back to the corresponding entry in the background information field, saved alongside the original four-tuple, and include the submitter's identifier, timestamp, and review status, forming a traceable iterative version.

[0082] For example, a one-click jump entry is embedded at the map anchor point. Clicking the jump entry calls the external interface of Google Maps or Amap, uses the latitude and longitude coordinates of the map anchor point as the destination parameter, and launches a visual route viewing page to present the walking or driving route from the user's current location to the coordinates of the multi-context sign.

[0083] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for perceptual analysis of differences in multi-contextual linguistic landscape symbols, characterized in that, The method includes: The image acquisition module captures real-world images of multi-context signs, and the positioning module binds geographical coordinates to anchor points on Google Maps and Gaode Maps. The signs are classified according to their appearance features and incorporated into the language landscape database. Optical character recognition is used to extract the outline of character strokes, identify the density of the arrangement and the uniformity of the width of the blank space, and determine the result of increased character density and the early threshold of language switching. A convolutional neural network is used to learn the features of increasing density sequences and output the character arrangement tomographic morphology. Candidate positions in the character arrangement tomographic morphology are screened, and language discrimination is performed to obtain the boundary region recognition result. The character segmentation module divides the stroke groups and verifies them with constraint conditions to determine the language boundary position. A gradient boosting decision tree is used to fuse the character arrangement fault morphology with the language boundary position to generate a boundary marker between the foreign language region and the Chinese region. Based on the boundary markers, the language of the signboard is labeled, and the language recognition result is output; Based on the sign type and language recognition results, retrieve the corresponding sign keywords, generate preliminary translations for each language, and write them into the translation field; The semantics of the initial translation are controlled, background cultural translations are added to the names of signs and attractions with extralinguistic background, translation strategy types are marked, and structured cultural annotation data is formed. Based on the structured cultural annotation data, spelling verification, grammar rule scanning, and semantic comparison of the translated text are performed on the signboard text. Error entries are written into the background information field, and language error status is marked on the map anchor points. The signboard information can be confirmed, corrected, added, and commented on by community users. Users can jump to Google Maps and Amap to view the visualized route with one click.

2. The method for perceiving differences in multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process involves capturing multi-context real-world images of signs using an image acquisition module, binding geographic coordinates to anchor points on Google Maps and Gaode Maps using a positioning module, classifying signs based on appearance features to determine their type, and incorporating this information into a comprehensive linguistic landscape database. This includes: The positioning module synchronously records the latitude and longitude coordinates of the shooting point, and the latitude and longitude coordinates are bound to the coordinate nodes of Google Maps and Gaode Maps to obtain a real-scene image of the sign with dual map anchor points; The appearance features of the signboard are extracted, including the shape of the signboard frame, the material of the base material, the posting method, and the graphic logo of the storefront. The signboard type is determined based on the appearance features and then incorporated into the language landscape database.

3. The method for perceiving differences in multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process involves using optical character recognition to extract character stroke outlines, identifying the density of character arrangement and the uniformity of white space width, and determining the result of increased character density and the early threshold for language switching, including: The character outline is delineated based on the connected components of the character stroke contour, and the scanning direction is set along the long side of the sign to obtain the character sequence; The density of character enclosing frames within a unit area is obtained by counting the number of such frames. The uniformity of white space width is obtained by measuring the width of the blank pixels between adjacent frames, calculating the standard deviation, and taking the reciprocal. The character density is increased by averaging the arrangement density using a fixed window length sliding window and multiplying it by the uniformity of the white space width position by position. The difference is then calculated along the scanning direction, and the position where the density increment exceeds a preset threshold is marked as the language switching early threshold.

4. The method for perceptual analysis of multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The method employs a convolutional neural network to learn the features of increasing density sequences and outputs the character arrangement tomographic morphology, including: The density-increasing sequence is extracted by extending a preset length to the left and right from the language switching early threshold, and then normalized to the maximum and minimum values ​​and padded with zero values ​​to a fixed length. The one-dimensional convolutional layer, max pooling layer, and fully connected layer of the convolutional neural network process the density increasing sequence in sequence, and output a fault morphology vector containing the density increase magnitude, density decrease magnitude, and fault straightness. The fault morphology vector is classified position by position into abrupt, gradual, or transitional fault morphology to obtain the character arrangement fault morphology.

5. The method for perceptual analysis of multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process of filtering candidate positions in the character arrangement tomographic morphology, determining the language to obtain the boundary region identification result, segmenting stroke groups by the character segmentation module and verifying them with constraints to determine the language boundary position includes: The coordinates of the locations located within a preset range near the language switching threshold and accompanied by abrupt or gradual fault patterns are selected to obtain a set of candidate locations. The outlines of the surrounding character strokes are extracted for each candidate location and fed into a random forest classifier. The language of the characters on the left and right sides is determined based on the stroke direction, stroke thickness ratio, and character height-to-width ratio to obtain the boundary region identification result. The character segmentation module reclassifies the connected components in the boundary area along the scanning direction, inserts a segmentation line where the stroke outline height or stroke thickness deviation exceeds a preset difference, and merges them to obtain foreign language stroke groups and Chinese stroke groups. Using the density of the arrangement and the uniformity of the width of the blank space as the constraints, the location where the difference in density of the arrangement on both sides of the junction between the foreign language stroke group and the Chinese stroke group exceeds a preset threshold and the uniformity of the width of the blank space is in the opposite direction is determined as the language boundary location.

6. The method for perceptual analysis of multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The step of using a gradient boosting decision tree to fuse the character arrangement fault morphology with the language boundary position to generate a boundary marker between the foreign language region and the Chinese region includes: Using the horizontal coordinate of the language boundary as the anchor point, the three components of density increase, density decrease and fault straightness within a preset length to the left and right of the anchor point are extracted and combined with the corresponding morphological labels to obtain the fused feature vector. A decision tree is built through multiple iterations. The leaf nodes output language affiliation scores. After weighted summation, positive values ​​represent Chinese regions and negative values ​​represent foreign language regions. Adjacent positions with opposite signs and crossing a preset zero threshold are determined as boundary points. The boundary marker between the foreign language region and the Chinese region is obtained by embedding segment labels.

7. The method for perceiving differences in multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The step of labeling the language of the signboard zones based on the boundary markers and outputting the language recognition result includes: Based on the horizontal coordinates of the endpoints of the segments in the boundary markers, the real-world image of the signboard is divided into several segments along the direction of the signboard, and the start and end ranges of the horizontal coordinates of each segment and the foreign language label or Chinese language label are obtained. Foreign language characters in the foreign language area are matched with foreign language labels based on optical character recognition, and Chinese characters in the Chinese language area are labeled as Chinese. The language recognition results are written into the language field of the language landscape comprehensive database.

8. The method for perceiving differences in multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process of retrieving corresponding sign keywords based on the sign type and language identification results, generating preliminary translations for each language, and writing them into the translation field includes: Based on the sign type, the corresponding sign keywords are retrieved from a pre-established venue function keyword database, where the venue function keyword database uses sign type as the key and Chinese sign keywords that directly convey the venue function as the value. For Chinese, English, Russian, Korean, Japanese, French, Spanish and Arabic, neural machine translation is used to translate the signboard keywords into multiple languages ​​with Chinese as the source language, and the preliminary translations in each language are written into the translation field.

9. The method for perceptual analysis of differences in multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process of controlling the semantics of the initial translation, supplementing the background cultural translations of the signs and attraction names with extralinguistic background information, and labeling the translation strategy type results in structured cultural annotation data, including: By comparing the preliminary translation with the pre-established cultural term index, the super-language background signs and scenic spot names that carry cultural connotations beyond the literal meaning are identified and the background cultural translations are supplemented to obtain a multilingual translation set with cultural translations. Each translation in the multilingual translation collection is labeled with a translation strategy type that is either phonetic-preserving, semantic-conversion, or hybrid. The cultural adaptation path is recorded in three segments: the original name, the conversion method, and the target language. The translated name entries, strategy tags, and path descriptions are combined to form a structured entry and stored in the database, thus forming the structured cultural annotation data.

10. The method for perceptual analysis of multi-contextual linguistic landscape symbols according to claim 1, characterized in that, The process involves performing spelling verification, grammar rule scanning, and semantic comparison of the translated text on the signboards based on the structured cultural annotation data. Error entries are identified and written into the background information field. Language error status is marked on map anchors. The system allows community users to confirm, correct, add, and comment on signboard information. Users can also jump to Google Maps and Gaode Maps with one click to view visualized routes. This includes: The extracted sign text is subjected to spelling verification by dictionary comparison, grammar rule scanning by grammar rule base matching, and semantic comparison of the translated text is performed based on the target language translation in the structured cultural annotation data to obtain the error type, the region, the original text and the suggested correction text, and the error entries are written into the background information field of the coordinate point image. The language error status is marked at the map anchor point according to the error entry, and the correction suggestions submitted by the open community users are written back to the background information field; The map anchor point embeds a one-click jump entry to the visualized route on Google Maps and Amap.

Images