A business hall service management method and an AR immersive introduction device

Abstract

The application provides a business hall service management method, which is realized through an AR immersive introduction device and comprises the following steps: when it is detected that there is a pedestrian in a specified area range, real-time image acquisition is performed; whether the pedestrian stops is detected according to the acquired real-time image; if it is detected that the pedestrian stops, information matching is performed on the stopping pedestrian in combination with a preset face vector information library, and after the matching is successful, the interesting content of the stopping pedestrian is generated and displayed and / or played. Through the application, different customer groups can be provided with specific service content, and the service content can be adjusted in time according to customer feedback.

Description

Technical Field

[0001] This invention relates to the field of circuit breaker fault detection technology, and in particular to a service management method for business halls and an AR immersive introduction device. Background Technology

[0002] With technological advancements, AR immersive informational devices have been adopted in the service halls of mobile and power companies to serve passersby. These devices utilize AR virtual digital service halls to replace traditional manual services. Compared to manual services, AR immersive informational devices can operate uninterruptedly while reducing the workload of staff.

[0003] However, existing AR immersive introduction devices struggle to tailor specific services to different customer groups and cannot adjust to customer feedback in a timely manner, resulting in a relatively simplistic overall service model. Therefore, to improve the overall service efficiency of AR immersive introduction devices, a more intelligent and comprehensive service management method for service halls is urgently needed. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a service management method for business halls and an AR immersive introduction device, which can not only realize the formulation of specific service content for different customer groups, but also adjust the service content in a timely manner based on customer feedback.

[0005] To address the aforementioned technical problems, this invention provides a service management method for business halls, which is implemented using an AR immersive introduction device. The method includes the following steps:

[0006] Real-time image acquisition is performed when pedestrians are detected within a designated area;

[0007] Based on the collected real-time images, detect whether there are pedestrians standing still;

[0008] If a pedestrian is detected standing still, the system will match the information of the standing pedestrian with the preset facial vector information database. Once a match is successful, the system will generate content of interest for the standing pedestrian and display and / or play it.

[0009] The step of detecting whether there are pedestrians standing still based on the acquired real-time images specifically includes:

[0010] Obtain the sampling interval, and according to the sampling interval, sample the acquired real-time image a certain number of times to generate multiple consecutive sampled images;

[0011] The Backbone network in deep learning is used to extract high-order human features from each sampled image, so that multiple human feature images are generated for each sampled image; each human feature image is represented by the human feature value through pixel value.

[0012] After denoising multiple person feature images generated from the same sampled image, they are then merged and transformed to obtain a one-dimensional vector of each sampled image;

[0013] Based on the one-dimensional vector of each sampled image, detect whether there are pedestrians standing still.

[0014] Among them, through the formula The sampling interval T is obtained; where P is the number of pedestrians sensed by the AR immersive introduction device;

[0015] Through formula Denoising is performed on multiple person feature images generated from the same sampled image; where f ij Let be the value at the j-th pixel on the feature image of the i-th person in a certain sampled image; α is a pre-set hyperparameter, which is a fixed value;

[0016] Through formula Multiple denoised feature images of people in the same sampled image are merged to form a pixel matrix F. ij And the pixel matrix F obtained by merging each sampled image ij Each is converted into a corresponding one-dimensional vector value; where n is the total number of personnel feature images in a certain sampled image.

[0017] The specific steps for detecting whether a pedestrian is standing based on the one-dimensional vector of each sampled image include:

[0018] Using a one-dimensional vector from three consecutive sampled images as a group, the formula is used... Calculate the overall one-dimensional vector value Dis among each group of one-dimensional vectors; where Vec1, Vec2, and Vec3 are the one-dimensional vector values ​​of three consecutive sampled images in the same group, respectively.

[0019] If Dis < γ, then a pedestrian is detected stopping; otherwise, if Dis ≥ γ, then no pedestrian is detected stopping; where γ is a pre-set hyperparameter, which is a fixed value.

[0020] The step of detecting a pedestrian stopping, matching the information of the stopping pedestrian with a preset facial vector information database, and generating and / or playing content of interest for the stopping pedestrian after a successful match specifically includes:

[0021] When a pedestrian is detected to be standing still, a pre-set multi-task convolutional neural network MTCNN is used to extract feature vectors from the face image of the standing pedestrian in order to obtain the face vector of the standing pedestrian.

[0022] If it is determined that the face vector of the pedestrian already exists in the face vector information database, the information matching is considered successful. Furthermore, based on the existing personnel characteristics in the face vector information database, the system maps and generates content of interest to the pedestrian in a preset service content recommendation system and displays and / or plays it. The content of interest to the pedestrian includes smart devices, broadband services, and family discount packages, and is displayed and / or played in a predetermined order.

[0023] The method further includes:

[0024] If the information matching is deemed unsuccessful, then in the face vector information database, based on the face vector of the stopped pedestrian, face vectors that meet the preset similarity conditions are retrieved, and further, based on the personnel characteristics of the face vectors retrieved in the face vector information database, the service content recommendation system maps and generates content of interest corresponding to the personnel characteristics of the retrieved face vectors and displays and / or plays it.

[0025] The method further includes:

[0026] When displaying and / or playing content of interest to the pedestrian, the confidence level of the pedestrian for each piece of content of interest is calculated, and the display and / or playback rate of each piece of content of interest is adjusted according to the confidence level of each piece of content of interest.

[0027] Specifically, the step of calculating the confidence level of the pedestrian for each piece of content of interest when displaying and / or playing the content of interest to the pedestrian includes:

[0028] Real-time image capture of the face of the pedestrian who stopped at each point of interest is performed during the display and / or playback of each piece of content of interest.

[0029] Using a pre-defined YOLOv8 network, facial keypoint detection is performed on the collected face images corresponding to each content of interest, resulting in a one-dimensional vector formed by all facial keypoints corresponding to each content of interest; wherein, the facial keypoints include four keypoints for the eyes, two keypoints for the ears, and a keypoint for the nose;

[0030] Using a preset Conv operator, convolution analysis is performed on the one-dimensional keypoint vectors corresponding to each content of interest to capture the regional relationship between each facial keypoint and obtain the first target mapping value of each content of interest; wherein, the first target mapping value of each content of interest is used to characterize the pedestrian's interest in the corresponding content of interest.

[0031] One-dimensional keypoint vectors corresponding to each content of interest at multiple time points are obtained and combined into a temporal vector. Furthermore, a preset Bi-LSTM algorithm is used to analyze the temporal vectors combined by each content of interest to obtain the second target mapping value of each content of interest.

[0032] Based on the first target mapping value of each content of interest and its corresponding second target mapping value, the confidence level of the pedestrian who stopped at the station for each content of interest is calculated.

[0033] Among them, through the formula ees total =δ*Res1+ε*Res2, calculate the confidence level Res of the pedestrian for each content of interest. total Where δ is the weight value, and ε is the weight value, and ε = 1 - δ; Res1 is the first target mapping value for each content of interest; Res2 is the second target mapping value for each content of interest; Conv rig The accuracy of predicting the first target mapping value Res1 in the Conv operator is a fixed value; BiLSTM rig This represents the accuracy of predicting the second target mapping value Res2 in the Bi-LSTM algorithm, and it is a fixed value.

[0034] Through formula Calculate the adjusted rate Sp for each content of interest; where l is the adjustment factor, which is a preset fixed value.

[0035] This invention also provides an AR immersive introduction device, comprising:

[0036] The pedestrian image acquisition unit is used to acquire images in real time when pedestrians are detected within a specified area.

[0037] The pedestrian detection unit is used to detect whether there are pedestrians standing still based on the real-time images collected.

[0038] The "Content of Interest" section is used to, if a pedestrian is detected standing still, combine the information of the standing pedestrian with a preset facial vector information database, and after a successful match, generate and display and / or play the content of interest for that standing pedestrian.

[0039] Implementing the embodiments of the present invention has the following beneficial effects:

[0040] 1. When a pedestrian is detected stopping, the present invention performs information matching. If successful, it generates content of interest for the stopping pedestrian and displays and / or plays it. Otherwise, if unsuccessful, it retrieves content of interest for people whose characteristics match preset similar conditions and displays and / or plays it. This allows for the customization of specific service content for different customer groups.

[0041] 2. This invention uses the Backbone network in deep learning to extract high-order features of people. After denoising, merging and transforming the people feature image formed after feature extraction, the detection of pedestrian standing is achieved by comparing the overall one-dimensional vector value with the threshold γ. This detection method not only speeds up the judgment of pedestrian standing, but also improves the accuracy.

[0042] 3. This invention uses a deep fusion network and a YOLOv8 algorithm network to perform feedback calculations on the key features of pedestrians who are stopping at a stop, thereby obtaining the confidence level of each pedestrian for each piece of content of interest. Based on the confidence level of each piece of content of interest, the display and / or playback rate of each piece of content of interest is adjusted, thereby enabling timely adjustment of service content based on customer feedback and significantly improving the overall effect of AR promotional videos. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0044] Figure 1 A flowchart illustrating a service management method for a business hall provided in an embodiment of the present invention;

[0045] Figure 2 A logical process diagram for calculating the overall offset value between one-dimensional vectors of three time-sampling images in an application scenario of a business hall service management method provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of a one-dimensional vector value of a sampled image at three different times in an application scenario of a service management method for a business hall provided in an embodiment of the present invention;

[0047] Figure 4 A flowchart illustrating the logical process of matching pedestrian information to generate content of interest in an application scenario of a service management method for a business hall provided in an embodiment of the present invention;

[0048] Figure 5A logical structure diagram of a preset YOLOv8 network in an application scenario of a business hall service management method provided in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of an AR immersive introduction device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown in the figure, a service management method for a business hall is proposed in an embodiment of the present invention. It is implemented through an AR immersive introduction device, and the method includes the following steps:

[0052] Step S1: When a pedestrian is detected within the specified area, real-time image acquisition is performed;

[0053] Step S2: Detect whether there are pedestrians standing still based on the collected real-time images;

[0054] Step S3: If a pedestrian is detected to be standing still, the information of the standing pedestrian is matched with the preset face vector information database. After the match is successful, the content of interest of the standing pedestrian is generated and displayed and / or played.

[0055] Specifically, in step S1, the AR immersive introduction device is placed in the sales hall, equipped with sensors, an image acquisition unit, a memory, and a CPU processor. At this time, the sensors detect a designated area (e.g., the angle directly in front)... + The system detects whether there are pedestrians in a fan-shaped area with a radius of 0.1 to 1.5 meters and detects the presence of pedestrians. When pedestrians are detected, the system uses an image acquisition device to collect images of pedestrians passing through the designated area. The acquisition frequency can be set to a low value, and the acquired images are stored.

[0056] In step S2, the AR immersive introduction device uses high-order feature extraction of personnel images and calculates whether the highlighted pixels have shifted, thereby determining whether there are pedestrians standing in front. Specifically, this includes:

[0057] First, through the formula To obtain the sampling interval T, and based on the sampling interval T, to sample a certain number of real-time images to generate multiple consecutive sampled images; where P is the number of pedestrians sensed by the AR immersive introduction device.

[0058] Secondly, the Backbone network in deep learning is used to extract high-order human features from each sampled image, so that multiple human feature images are generated in each sampled image; each human feature image is characterized by human feature values ​​through pixel values.

[0059] In one example, a mid-layer VGG network architecture within the Backbone network is used to extract high-order human features from each sampled image. In this case, the image may contain high-order human-related information points, which are represented by higher pixel values ​​in the human feature map.

[0060] Next, after denoising multiple personnel feature images generated from the same sampled image, they are merged and transformed to obtain a one-dimensional vector of each sampled image.

[0061] In one example, through the formula Denoising is performed on multiple person feature images generated from the same sampled image; where f ij Let be the value at the j-th pixel on the feature image of the i-th person in a certain sampled image; α is a pre-set hyperparameter, which is a fixed value;

[0062] Through formula Multiple denoised feature images of people in the same sampled image are merged to form a pixel matrix F. ij And the pixel matrix F obtained by merging each sampled image ij Each is converted into a corresponding one-dimensional vector value; where n is the total number of personnel feature images in a certain sampled image.

[0063] At this point, the Feature Map follows the threshold formula f mentioned above. ij The process involves filtering out irrelevant pixel values ​​and setting them to 0 (e.g., background, to eliminate the offset effects caused by other elements) to avoid numerical jitter affecting subsequent calculations. Then, all channels of each filtered feature map are merged according to an addition formula, i.e., each group of feature map channels is added together using an Add method to sum the values ​​on all feature maps. Finally, a matrix-to-vector transformation is performed.

[0064] Finally, based on the one-dimensional vectors of each sampled image, it is determined whether a pedestrian is standing still. Specifically, the one-dimensional vectors of three consecutive sampled images are grouped together, and the result is obtained using the formula... Calculate the overall one-dimensional vector value Dis among each group of one-dimensional vectors; where Vec1, Vec2, and Vec3 are the one-dimensional vector values ​​of three consecutive sampled images in the same group; if Dis < γ, then a pedestrian is detected standing; otherwise, if Dis ≥ γ, then no pedestrian is detected standing; where γ is a pre-set hyperparameter, which is a fixed value.

[0065] It should be noted that a group of one-dimensional vectors from three consecutive sampled images can be obtained by equally dividing the one-dimensional vectors of all sampled images, or by sequentially polling the one-dimensional vectors of each sampled image.

[0066] In one example, sampled images are acquired at three time points. These images are sampled consecutively at sampling intervals T and converted into three final one-dimensional vectors. Using the formula Dis, the features of the person vectors at these three different time points can be compared. Figure 2 As shown.

[0067] If a pedestrian stops, the vectors at these three moments will be highly similar, resulting in a smaller Dis value. When Dis is less than γ, it is determined that the pedestrian has stopped. However, if no pedestrian stops, meaning the pedestrian continues walking without stopping, this phenomenon is reflected in the one-dimensional vector as a small change in the overall value, but a significant change in position. Figure 3 The overall height of the one-dimensional vector remains unchanged, but the position changes, and thus the value of Dis increases. That is, when it is greater than or equal to γ, it is determined that no pedestrian is standing.

[0068] In step S3, the AR immersive introduction device pre-stores a facial vector information database and a service content recommendation system. When it detects a pedestrian stopping, it uses a pre-defined multi-task convolutional neural network (MTCNN) to extract feature vectors from the pedestrian's facial image to obtain the pedestrian's facial vector. It should be noted that MTCNN for facial feature vector extraction is a commonly used technique in this field and will not be elaborated upon here.

[0069] If it is determined that the face vector of the pedestrian already exists in the face vector information database, the information matching is considered successful. Furthermore, based on the existing personnel characteristics in the face vector information database, the service content recommendation system maps and generates content of interest to the pedestrian and displays and / or plays it. The content of interest to the pedestrian includes smart devices, broadband services and family discount packages, and is displayed and / or played in a predetermined order.

[0070] If it is determined that the face vector of the pedestrian who is standing does not exist in the face vector information database, the information matching is considered unsuccessful. In the face vector information database, based on the face vector of the pedestrian who is standing, face vectors that meet the preset similarity conditions are retrieved. Furthermore, based on the personnel characteristics of the face vectors retrieved in the face vector information database, the service content recommendation system maps and generates content of interest corresponding to the personnel characteristics of the retrieved face vector and displays and / or plays it.

[0071] It should be noted that a corresponding mapping relationship is established between the facial vector information database and the service content recommendation system based on personnel characteristics. Preset similarity conditions can be flexibly designed according to actual needs, such as using gender and age range as keywords for fuzzy matching.

[0072] In one example, such as Figure 4 As shown, if the facial vector information of the pedestrian matches the facial vector database, the service content recommendation system will generate content that the pedestrian is interested in. For example, if the pedestrian's interests are smart devices, broadband services, and family discount packages, the playback order will be ranked as smart devices, broadband services, and family discount packages.

[0073] If the information is not found, the system returns the features (including gender and age) of the person closest to the retrieved face vector. Based on these features, the system then maps and generates content of interest for the person corresponding to the retrieved face vector in the service content recommendation system. For example, if the person is male and 18 years old, the service content recommendation system might suggest that an 18-year-old male might be particularly interested in the latest smart devices in the service hall.

[0074] In this embodiment of the invention, the AR immersive introduction device uses a deep fusion network and the YOLOv8 algorithm to collect key feature feedback from pedestrians. It calculates the confidence level of each piece of content of interest to the pedestrian to provide service quality feedback, and adjusts the video rate based on this confidence level. Therefore, the method further includes: when displaying and / or playing content of interest to the pedestrian, calculating the pedestrian's confidence level for each piece of content of interest, and adjusting the display and / or playback rate of each piece of content of interest based on its confidence level.

[0075] At this point, the AR immersive introduction device first captures real-time images of the pedestrian's face during the display and / or playback of each piece of interest. Second, using a pre-defined YOLOv8 network, it performs facial landmark detection on the captured face images corresponding to each piece of interest, obtaining a one-dimensional vector formed by all facial landmarks for each piece of interest. Next, using a pre-defined Conv operator, it performs convolution analysis on the one-dimensional landmark vectors corresponding to each piece of interest to capture the regional relationships between the facial landmarks, obtaining the first target mapping value for each piece of interest. Then, it acquires the one-dimensional landmark vectors corresponding to each piece of interest at multiple time points and combines them into a temporal vector. Furthermore, it uses a pre-defined Bi-LSTM algorithm to analyze the temporal vectors combined by each piece of interest, obtaining the second target mapping value for each piece of interest. Finally, based on the first target mapping value and its corresponding second target mapping value for each piece of interest, it calculates the target mapping value using the formula Res. total =δ*Res1+ε*Res2, calculate the confidence level Res of the pedestrian for each content of interest. total And further through formula Calculate the adjusted rate Sp for each content of interest. Here, l is the adjustment factor, which is a preset fixed value; δ is the weight value, and... ε is the weight value, and ε = 1 - δ; Res1 is the first target mapping value for each content of interest; Res2 is the second target mapping value for each content of interest; Conv rig The accuracy of predicting the first target mapping value Res1 in the Conv operator is a fixed value; BiLSTM rig This represents the accuracy of predicting the second target mapping value Res2 in the Bi-LSTM algorithm, and it is a fixed value.

[0076] It should be noted that facial key points include four key points for the eyes, two key points for the ears, and a key point for the nose; the first target mapping value for each content of interest is used to characterize the pedestrian's interest in the corresponding content of interest.

[0077] Understandably, deep fusion networks and YOLOv8 algorithms are used to perform feedback calculations on key user features to verify whether the user is interested in the current content. Weights are adjusted based on the accuracy of each network layer to increase the overall network's fault tolerance and computation speed.

[0078] In one example, such as Figure 5 As shown, when displaying and / or playing content of interest, the YOLOv8 network is used to detect facial key points, including four key points for the eyes, two key points for the ears, key points for the nose, etc., and all facial key points in each frame are recorded and further fed back to form a one-dimensional vector formed by all facial key points.

[0079] Unlike previous keypoint identification and analysis methods, this approach uses the Conv operator to perform convolution analysis on the one-dimensional vector corresponding to the currently interesting content, in order to capture the regional relationships between these feature points. The Conv algorithm is set to 1*3 and padded with 1s. The final target mapping value Res1 represents whether the pedestrian is interested in the content and their level of attention to the service presentation. In actual labeling, if the pedestrian does not turn their head, move, or look at their phone, the label is 1, indicating a high level of attention to the service presentation.

[0080] In addition, a temporal vector is formed by combining one-dimensional vectors of the content of interest at different times, and the Bi-LSTM algorithm is used to analyze the temporal vectors of the content of interest. The one-dimensional vector at each time is used as an input value. Ten sets of one-dimensional vectors are collected as a set of input values ​​for the Bi-LSTM network. At the end of the network, a fully connected FC layer is used to output the second target mapping value Res2.

[0081] At this point, using the formula Res total =δ*Res1+ε*Res2, calculate the confidence level Res of the pedestrian who is currently interested in the content. total Therefore, the weights can be balanced based on the accuracy of the Conv operator and the Bi-LSTM algorithm. Thus, when the Conv operator's prediction accuracy is high, Res1 contributes more to the final result; conversely, Res1 contributes less to the final prediction result.

[0082] Based on the confidence level Res of the currently interested content mentioned above total Through formula The playback speed can be adjusted based on the level of interest of pedestrians in the currently engaging video content. When the interest level is low, the AR immersive presentation device accelerates the playback of the video content. For example, when the initial presentation content is about smart devices, if the customer's interest level is low, the video content will be played at a faster speed. This speed will return to normal once the pedestrian's interest level increases.

[0083] like Figure 6 As shown in the figure, an AR immersive introduction device is proposed in an embodiment of the present invention, comprising:

[0084] The pedestrian image acquisition unit 110 is used to acquire images in real time when a pedestrian is detected within a specified area.

[0085] The pedestrian detection unit 120 is used to detect whether there are pedestrians standing still based on the real-time images collected.

[0086] The content of interest introduction unit 130 is used to, if a pedestrian is detected to be standing still, combine the preset face vector information database to perform information matching on the standing pedestrian, and after a successful match, generate the content of interest of the standing pedestrian and display and / or play it.

[0087] Implementing the embodiments of the present invention has the following beneficial effects:

[0088] 1. When a pedestrian is detected stopping, the present invention performs information matching. If successful, it generates content of interest for the stopping pedestrian and displays and / or plays it. Otherwise, if unsuccessful, it retrieves content of interest for people whose characteristics match preset similar conditions and displays and / or plays it. This allows for the customization of specific service content for different customer groups.

[0089] 2. This invention uses the Backbone network in deep learning to extract high-order features of people. After denoising, merging and transforming the people feature image formed after feature extraction, the detection of pedestrian standing is achieved by comparing the overall one-dimensional vector value with the threshold γ. This detection method not only speeds up the judgment of pedestrian standing, but also improves the accuracy.

[0090] 3. This invention uses a deep fusion network and a YOLOv8 algorithm network to perform feedback calculations on the key features of pedestrians who are stopping at a stop, thereby obtaining the confidence level of each pedestrian for each piece of content of interest. Based on the confidence level of each piece of content of interest, the display and / or playback rate of each piece of content of interest is adjusted, thereby enabling timely adjustment of service content based on customer feedback and significantly improving the overall effect of AR promotional videos.

[0091] It is worth noting that in the above device embodiments, the various device units included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0092] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as ROM / RAM, disk, optical disk, etc.

[0093] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A service management method for a business hall, which is implemented through an AR immersive introduction device, characterized in that, The method includes the following steps: Real-time image acquisition is performed when pedestrians are detected within a designated area; A sampling interval is obtained, and a certain number of samples are taken from the acquired real-time image according to the sampling interval to generate multiple consecutive sampled images. A Backbone network in deep learning is used to perform high-order extraction of human features from each sampled image, so that multiple human feature images are generated corresponding to each sampled image. Each human feature image represents the human's feature value through pixel values. After denoising the multiple human feature images generated in the same sampled image, they are merged and transformed to obtain a one-dimensional vector for each sampled image. Based on the one-dimensional vectors of each sampled image, it is detected whether a pedestrian is standing still. If a pedestrian is detected to be standing still, the information of the standing pedestrian is matched with the preset face vector information database. After a successful match, the content of interest of the standing pedestrian is generated and displayed and / or played. The method further includes: when displaying and / or playing content of interest to the pedestrian, calculating the pedestrian's confidence level for each piece of content of interest, and adjusting the display and / or playback rate of each piece of content of interest according to the confidence level of each piece of content of interest; The step of calculating the confidence level of the pedestrian for each piece of content of interest when displaying and / or playing the content of interest to the pedestrian specifically includes: Real-time image capture of the face of the pedestrian who stopped at each point of interest is performed during the display and / or playback of each piece of content of interest. Using a pre-defined YOLOv8 network, facial keypoint detection is performed on the collected face images corresponding to each content of interest, resulting in a one-dimensional vector formed by all facial keypoints corresponding to each content of interest; wherein, the facial keypoints include four keypoints for the eyes, two keypoints for the ears, and a keypoint for the nose; Using a preset Conv operator, convolution analysis is performed on the one-dimensional keypoint vectors corresponding to each content of interest to capture the regional relationship between each facial keypoint and obtain the first target mapping value of each content of interest; wherein, the first target mapping value of each content of interest is used to characterize the pedestrian's interest in the corresponding content of interest. One-dimensional keypoint vectors corresponding to each content of interest at multiple time points are obtained and combined into a temporal vector. Furthermore, a preset Bi-LSTM algorithm is used to analyze the temporal vectors combined by each content of interest to obtain the second target mapping value of each content of interest. Based on the first target mapping value of each content of interest and its corresponding second target mapping value, the confidence level of the pedestrian who stopped at the station for each content of interest is calculated.

2. The service management method for business halls as described in claim 1, characterized in that, Through formula The sampling interval is obtained. ;in, The number of pedestrians sensed by the AR immersive introduction device; Through formula Denoising is performed on multiple person feature images generated from the same sampled image; among them, For a certain sampled image, the first The first person's feature image The value at each pixel; These are pre-set hyperparameters, and they are fixed values. Through formula Multiple denoised feature images of people in the same sampled image are merged to form a pixel matrix. And merge the pixel matrix obtained by combining each sampled image. All are converted into corresponding one-dimensional vector values; where, for Total quantity.

3. The service management method for business halls as described in claim 2, characterized in that, The specific steps for detecting whether a pedestrian is standing based on the one-dimensional vector of each sampled image include: Using a one-dimensional vector from three consecutive sampled images as a group, the formula is used... Calculate the overall one-dimensional vector value between each group of one-dimensional vectors. ;in, , , A one-dimensional vector value for three consecutive sampled images in the same group; like If the signal is positive, then a pedestrian is detected stopping; otherwise, if the signal is negative, the pedestrian stops stopping. If the detection detects that no pedestrians stopped, then no pedestrians were found to be standing; among them, These are pre-set hyperparameters, and they are fixed values.

4. The service management method for business halls as described in claim 1, characterized in that, The step of detecting a pedestrian stopping, matching the information of the stopping pedestrian with a preset facial vector information database, and generating and / or playing content of interest for the stopping pedestrian after a successful match specifically includes: When a pedestrian is detected to be standing still, a pre-set multi-task convolutional neural network MTCNN is used to extract feature vectors from the face image of the standing pedestrian in order to obtain the face vector of the standing pedestrian. If it is determined that the face vector of the pedestrian already exists in the face vector information database, the information matching is considered successful. Furthermore, based on the existing personnel characteristics in the face vector information database, the system maps and generates content of interest to the pedestrian in a preset service content recommendation system and displays and / or plays it. The content of interest to the pedestrian includes smart devices, broadband services, and family discount packages, and is displayed and / or played in a predetermined order.

5. The service management method for business halls as described in claim 4, characterized in that, The method further includes: If the information matching is deemed unsuccessful, then in the face vector information database, based on the face vector of the stopped pedestrian, face vectors that meet the preset similarity conditions are retrieved, and further, based on the personnel characteristics of the face vectors retrieved in the face vector information database, the service content recommendation system maps and generates content of interest corresponding to the personnel characteristics of the retrieved face vectors and displays and / or plays it.

6. The service management method for business halls as described in claim 1, characterized in that, Through formula Calculate the confidence level of the pedestrian who stopped for each item of interest. ;in, The weight value, and ; The weight value, and ; The first target mapping value for each content of interest; The second target mapping value for each content of interest; Predict the first target mapping value in the Conv operator The accuracy rate is a fixed value; Predicting the second target mapping value in the Bi-LSTM algorithm The accuracy rate is a fixed value; Through formula Calculate the adjusted rate for each content of interest. ;in, The adjustment factor is a preset fixed value.

7. An AR immersive introduction device, characterized in that, include: The pedestrian image acquisition unit is used to acquire images in real time when pedestrians are detected within a specified area. A pedestrian detection unit is used to obtain a sampling interval and, based on the sampling interval, sample a certain number of times the acquired real-time image to generate multiple consecutive sampled images. A Backbone network from deep learning is used to perform high-order extraction of human features from each sampled image, so that multiple human feature images are generated corresponding to each sampled image. Each human feature image represents the human's feature value through pixel values. After denoising the multiple human feature images generated from the same sampled image, they are merged and transformed to obtain a one-dimensional vector for each sampled image. Based on the one-dimensional vectors of each sampled image, the unit detects whether a pedestrian is standing still. The "Content of Interest" section is used to, if a pedestrian is detected standing still, combine the information of the standing pedestrian with a preset facial vector information database, and after a successful match, generate and display and / or play the content of interest of the standing pedestrian. When displaying and / or playing content of interest for the pedestrian, the content of interest introduction unit calculates the pedestrian's confidence level for each piece of content of interest, and adjusts the display and / or playback rate of each piece of content of interest based on the confidence level of each piece of content of interest. Specifically, when displaying and / or playing content of interest to the pedestrian, calculating the pedestrian's confidence level for each piece of content of interest includes: Real-time image capture of the face of the pedestrian who stopped at each point of interest is performed during the display and / or playback of each piece of content of interest. Using a pre-defined YOLOv8 network, facial keypoint detection is performed on the collected face images corresponding to each content of interest, resulting in a one-dimensional vector formed by all facial keypoints corresponding to each content of interest; wherein, the facial keypoints include four keypoints for the eyes, two keypoints for the ears, and a keypoint for the nose; Using a preset Conv operator, convolution analysis is performed on the one-dimensional keypoint vectors corresponding to each content of interest to capture the regional relationship between each facial keypoint and obtain the first target mapping value of each content of interest; wherein, the first target mapping value of each content of interest is used to characterize the pedestrian's interest in the corresponding content of interest. One-dimensional keypoint vectors corresponding to each content of interest at multiple time points are obtained and combined into a temporal vector. Furthermore, a preset Bi-LSTM algorithm is used to analyze the temporal vectors combined by each content of interest to obtain the second target mapping value of each content of interest. Based on the first target mapping value of each content of interest and its corresponding second target mapping value, the confidence level of the pedestrian who stopped at the station for each content of interest is calculated.

Images