Automatic vending system for tourist area and control method

Abstract

The invention provides an automatic vending system for a tourist area. The automatic vending system comprises a vending device, a camera device and a system platform, the camera device is mounted in the tourism area; the camera device is connected with the system platform and sends detected data of a purchase intention to the system platform; the system platform is connected with the vending device so as to control the vending device to go to tourists to sell goods, and the problem that the tourists are inconvenient to purchase goods in a tourism area is solved. A camera device in a tourist area is fully utilized to detect and collect tourist images in real time, when it is detected that some tourists do some limb actions, it is judged that the tourists have the intention to purchase the tourists, the positions of the tourists are detected, and then the selling trolley is controlled to go to the positions of the tourists for selling goods. The problems that tourists need to walk along a long route when purchasing goods, much physical strength and time are consumed, even some tourists do not know where goods selling points are located, and the tourists need to look for goods slowly for shopping in a tourist area are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of tourist area shopping, and in particular to an automatic vending system and a control method for tourist areas. Background Art

[0002] With the development of social economy, tourism has become a way of life for the public, including long-distance travel, short-distance travel, and weekend park tours. Regardless of the type of travel, most people will visit managed scenic spots. At the same time, in order to facilitate tourists to buy items, the management of the tourist area generally sets up one or two convenience stores in the tourist area.

[0003] With the development of technology and the requirements for shopping convenience and cost reduction, convenience stores have gradually developed into vending machines. For example, the Chinese utility model application number CN202120608701.9 discloses a tourist souvenir vending device, and the Chinese invention application number CN201611210660.8 discloses a tourist WIFI vending machine, both of which are designed to facilitate the purchase of goods during travel.

[0004] However, vending machines or convenience stores in tourist areas are often designed in fixed locations. Tourists have to go to fixed locations to buy things when they want to buy something. This is not only tiring, but also takes up tourists’ time. It not only consumes physical energy but also reduces tourists’ play time. Some tourists even don’t know where the sales point is and need to look for it slowly. Therefore, shopping is not only inconvenient but also reduces tourists’ travel experience. At the same time, if you order takeout in a tourist area, not only will the extra cost be high, but the waiting time will be long.

[0005] In view of this, there is a need for an automatic vending system and control method for a tourist area. Summary of the invention

[0006] In view of the inconvenience of shopping in tourist areas and the reduction of tourists' travel experience in the prior art, the present invention provides an automatic vending system and control method for tourist areas, which can maintain tourists' travel experience and cost while increasing the convenience of tourists' shopping. The specific technical solution is as follows:

[0007] An automatic vending system for a tourist area comprises: a vending device, a camera device and a system platform; the camera device is installed in the tourist area and is used to detect tourists' intention to purchase goods; the camera device is connected to the system platform and sends data of the detected purchase intention to the system platform; the system platform is connected to the vending device to control the vending device to go to tourists to sell goods, so as to solve the problem of inconvenience for tourists to purchase goods in the tourist area.

[0008] Furthermore, the vending device includes an automatic vending cart or a drone.

[0009] Furthermore, the vending cart includes a wheeled type or a track type, and the track type is faster and safer.

[0010] Furthermore, the system platform includes an application server and a database server; the application server is connected to the database server, the vending device and the camera device respectively.

[0011] Furthermore, the camera device adopts a high-definition camera; the camera is installed on a pole set up in the tourist area.

[0012] A method for controlling an automatic vending system in a tourist area, applied to the automatic vending system in the tourist area described above, comprising:

[0013] The camera device detects and collects the images of tourists in real time;

[0014] The system platform analyzes tourist images to determine their purchasing intentions;

[0015] If the system platform determines that the tourist has the intention to purchase goods, the vending device is controlled to go to the tourist to sell goods.

[0016] Furthermore, the system platform analyzes the tourist image to determine the tourist's purchasing intention, including analyzing whether the tourist image has specific body movements. If the tourist makes specific body movements, it is determined that the tourist has purchasing intention.

[0017] Furthermore, the specific body movements are displayed to inform tourists when they enter the park or in the recreational area of ​​the tourist area.

[0018] Furthermore, the control of the vending device to go to the tourists to sell goods includes the following steps:

[0019] The location and installation angle of the camera device that detects the tourists' purchasing intention;

[0020] Determine the first area where the tourist is located according to the position and installation angle of the camera device;

[0021] According to the location of the vending device, a path for the vending device to go to the first area is planned, and the vending device is controlled to go to and enter the first area;

[0022] The distance between the vending device and the tourist is detected, and the vending device is controlled to move to a preset distance from the tourist according to the distance.

[0023] Furthermore, the method of controlling the vending device to go to the tourists to sell goods also includes the following steps:

[0024] The image data of tourists who intend to buy goods are saved, and the various cameras in the tourist area are used to relay the detection of the tourists, so as to detect the second area where the tourists are located in real time, and control the vending device to go to the second area. This solution allows the system to know the location of tourists in real time. Tourists do not have to wait in a fixed place like waiting for takeout. Tourists can wait for the vending cart to find them while playing, so as to maintain the playing experience and shopping experience.

[0025] Furthermore, when a tourist's intention to buy is detected, a drone can be used to fly to the tourist's location, and the tourist can be asked what he wants to buy by voice collection, and then a sales cart can be controlled to go to the tourist's location.

[0026] Furthermore, the specific body movements of tourists can be further detected to determine the types of items purchased by tourists, and then the corresponding sales carts can be controlled to sell the items.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. Make full use of the camera equipment in the tourist area to detect and collect tourist images in real time. When it is detected that some tourists make certain body movements, it is determined that the tourist has the intention to buy, and the location of the tourist is detected. Then, the sales cart is controlled to go to the location of the tourist to sell goods, so as to solve the problem that tourists have to take a long route to buy goods, which consumes more physical strength and time. Some tourists even don’t know where the sales point is and need to slowly look for it, etc., which is inconvenient for shopping in tourist areas.

[0029] 2. The sales cart can be four-wheeled or track-based. The track-based one is faster and safer.

[0030] 3. It can maintain tourists’ travel experience and costs while increasing their shopping convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0032] Figure 1 It is a structural schematic diagram of an automatic vending system in a tourist area;

[0033] Figure 2 The figure is a flow chart of a control method for an automatic vending system in a tourist area. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0036] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0037] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] Embodiment 1

[0039] like Figure 1 The figure shows a schematic diagram of the structure of an automatic vending system in a tourist area, comprising: a vending device, a camera device and a system platform; the camera device is installed in the tourist area and is used to detect tourists' purchasing intentions; the camera device is connected to the system platform and sends data on the detected purchasing intentions to the system platform; the system platform is connected to the vending device to control the vending device to go to tourists to sell goods, so as to solve the problem of inconvenience for tourists in purchasing goods in the tourist area.

[0040] Furthermore, the vending device includes an automatic vending cart or a drone.

[0041] Furthermore, the vending cart is a small mobile device that can automatically vend goods. The following is a detailed introduction about it:

[0042] 1. Basic structure

[0043] Usually made of strong and lightweight materials, such as metal (aluminum alloy, etc.) or high-strength plastic. Metal shells can provide better protection and prevent external forces from damaging the equipment inside the vehicle; plastic shells are lighter, which helps to reduce the weight of the vehicle and facilitate mobility. They are also more diverse in appearance design and can be customized according to different scenarios and brand images.

[0044] There are multiple layers of shelves and warehouses for storing various goods for sale. The design of these storage areas will vary depending on the type of goods sold. For example, if it is selling beverages, there will be special grids to place canned or bottled beverages of different sizes, and there will be devices to prevent the beverages from shaking and tipping over; if it is selling snacks, the height and spacing of the shelves will be adjusted according to the size of the snack packaging to make full use of the space.

[0045] Payment devices: including cash receiving devices (such as banknote validators and coin changers) and electronic payment devices (such as barcode scanners and NFC card readers). Banknote validators can quickly and accurately identify the authenticity and denomination of banknotes, while coin changers can automatically make changes based on the price of the goods and the amount paid by the user. Electronic payment devices facilitate consumers to pay with their mobile phones. Barcode scanners can read mobile payment codes, while NFC card readers support payment using payment cards or mobile phones with NFC functions.

[0046] Goods delivery device: generally realized through a mechanical transmission structure. When the consumer completes the payment, the delivery device will push the selected goods to the pickup port. For example, some vending carts use a spiral delivery device to push the goods out of the shelf by rotating the spiral rod; others use a conveyor belt delivery device, where the goods are placed on the conveyor belt, and after the payment is successful, the conveyor belt transports the goods to the pickup port.

[0047] Electric drive: Most vending carts are electric, equipped with a battery pack (such as lithium battery) and a motor. The battery pack provides power for the entire operation of the cart, including movement and operation of the vending system. The motor drives the wheels to rotate, allowing the cart to travel on a preset route or move within a designated area. This electric drive method is energy-saving and environmentally friendly, and it has low noise during operation.

[0048] There is a controller, which is the "brain" of the vending cart, usually composed of a microprocessor and related software. The controller can manage the sales process, such as recording commodity inventory, processing payment transactions, etc. At the same time, it can also control the movement of the cart and reach the designated area for patrol sales according to the calculated route. The controller can also be connected to the remote monitoring system, and merchants can remotely view the operation status, inventory status and other information of the cart through mobile applications or web pages.

[0049] The cart has a transparent window or display screen to display product information. The transparent window allows consumers to intuitively see the types and packaging of products in the cart, attracting them to buy. Some advanced vending carts are also equipped with electronic display screens that can loop product advertisements, price information, promotional activities and other content to achieve better marketing effects.

[0050] This approach has the following advantages:

[0051] Flexible sales: Compared with traditional fixed sales points, the vending cart can flexibly adjust its position according to factors such as the flow of people and time. It can move between different areas and actively look for potential consumers, thereby expanding the sales scope. For example, at different times of the day, the cart can adjust its route and stop position according to the flow direction and gathering area of ​​the crowd, such as staying near places with large flow of people during peak hours.

[0052] The cost is relatively low. Opening a traditional physical store requires a series of expenses such as renting a store, decorating, and hiring employees. The initial investment of a vending car is mainly concentrated on equipment procurement and simple maintenance, and there is no need to pay high rent and a large amount of labor costs. Although regular replenishment and maintenance of equipment are required, the overall operating cost is relatively low, which is a more economical way of selling for some small businesses or startups.

[0053] 24-hour operation potential: As long as there is sufficient inventory and power supply, the vending car can operate 24 hours a day. In places with frequent personnel flow, such as hospitals and railway stations, it can provide consumers with basic commodity sales services at night to meet people's urgent needs.

[0054] In the future, vending machines may be equipped with more advanced artificial intelligence technologies, such as intelligent voice interaction systems. Consumers can use voice commands to query product information and complete purchase operations. At the same time, the car's controller can optimize sales routes and product configurations through big data analysis. According to consumption data from different places, the types of goods and inventory quantities in the car can be automatically adjusted to improve sales efficiency.

[0055] Expanding service functions, in addition to selling traditional goods, vending carts also have some value-added services. For example, in some tourist attractions, the cart can provide tour guide services (through voice equipment) and sell tourist souvenirs; in the community, the cart can provide simple life services, such as collecting express deliveries, printing documents, etc.

[0056] Furthermore, existing vending car models include Huawo Zhiban HW-PS01, Zhonggong Yiduoxing WWW5030XSHE6 mobile vending car, Chengli CL5031XDW6BZ mobile service car, etc.

[0057] Furthermore, the vending cart can be equipped with a positioning module, a voice collection device and a sound amplification device based on the above scheme; the voice collection device, the sound amplification device and the positioning module are respectively connected to the controller of the cart to collect the voice questions of tourists, and the positioning position and the questions are fed back to the system platform of the whole system through the controller; the system platform generates corresponding answers based on the positioning position and the questions by using the intelligent model and feeds them back to the controller, and the controller then feeds back to the tourists through the sound amplification device, so that it is convenient to answer the questions of tourists at any time, improve the travel experience of tourists and also reduce the management cost of the tourist area.

[0058] Furthermore, the specific internal structures of the positioning module, voice collection equipment, sound amplification equipment and controller can adopt existing products or technical solutions as long as the same functions are achieved.

[0059] Furthermore, the vending cart may also include a track-type one, which is faster and safer.

[0060] Furthermore, the vending device may also use a drone, which can move in a straight line without being restricted by terrain, so that the delivery can be achieved at a higher speed, and it can be used to quickly deliver emergency medicines when a person is injured. Furthermore, the drone is provided with a device for delivering goods or medicines, which is used to deliver goods or medicines without contacting the ground. If the goods delivered are fragile, they can be delivered as close to the ground as possible. The specific delivery device structure can adopt the existing technical solutions.

[0061] Furthermore, the camera device uses a high-definition camera; the camera is installed on a pole set up in the tourist area; the pole can be set near the road in the tourist area and near the recreational area to cover all people as much as possible while saving costs as much as possible.

[0062] Camera selection: Choose the right camera according to the different detection scenarios. In indoor scenic spots and venues (such as museums and exhibition halls), you can choose high-definition, low-light network cameras to ensure that portraits can be clearly captured even in darker indoor environments. For example, some cameras with infrared night vision functions can obtain high-quality portrait images even in the corners of exhibition halls without lights. In outdoor scenic spots (such as natural scenery scenic spots and theme parks), it is necessary to consider waterproof, sun-proof, and dust-proof cameras, and their focal lengths must be able to meet the requirements of covering the corresponding areas. For example, for vast squares or large amusement facilities areas, cameras with telephoto lenses should be used to ensure that clear portraits of distant tourists can be captured.

[0063] The resolution of the camera is also a key factor. A high-resolution camera (such as a 4K camera) can provide clearer and more detailed portraits, which is very beneficial for subsequent operations such as feature recognition. However, the pressure of data storage and transmission must also be considered, because the amount of data in high-resolution images is large.

[0064] Camera layout and installation: For key locations such as the entrance, exit, and main passages of the scenic area, ensure full coverage of the camera. For example, at the entrance of the scenic area, multiple cameras can be installed to shoot the tourists entering from different angles (front and side) to obtain complete portrait information. In some narrow passages, such as aisles between ancient buildings, the position of the camera should be arranged reasonably to avoid blind spots in monitoring. The installation height and angle also need to be carefully designed. Generally speaking, the installation height of the camera should be between 2.5-3.5 meters. This height can effectively avoid being touched by tourists and at the same time obtain a better shooting angle. In terms of angle, try to ensure that the camera can shoot the front or side of the tourists, because these angles are most beneficial for portrait recognition and feature extraction.

[0065] Furthermore, the system platform includes an application server and a database server; the application server is connected to the database server, the vending device and the camera device respectively; the application server is provided with application programs set up as needed, such as a tourist image limb feature recognition model and a natural language processing model.

[0066] In the specific implementation, the application server is configured with software that can obtain camera images in real time. The software should have a stable image stream receiving function and be able to adapt to various models of cameras. For example, it can communicate with network cameras of different brands through the ONVIF (Open Network Video Interface Forum) protocol to ensure stable image transmission, and can also be carried out through other protocols with the same function. The software also needs to have image preprocessing functions, such as automatically adjusting the brightness, contrast and color balance of the image. Under different lighting conditions (such as outdoor attractions with strong light and darker indoor attractions), preprocessing can make the quality of the collected portrait images more consistent, which is convenient for subsequent detection. The software is listed but not limited to the following: Amcap is a free installation camera software that provides a simple and easy-to-use interface, supports the identification and selection of multiple camera devices, and can provide stable and smooth camera image preview and screenshot functions without occupying too many system resources. It is suitable for industrial automation, security monitoring, scientific research experiments, education and training and other scenarios. Live Camera Viewer: It has excellent performance optimization, can make full use of hardware resources, and ensures efficient operation on mobile devices. Its interface design is simple and clear, and the operation is smooth. It supports advanced functions such as multi-camera management, flexible recording and screenshot functions, and real-time alarm. Other software with the same functions can also be used.

[0067] Considering that you may need to process images from multiple cameras at the same time, you need to allocate system resources reasonably. For the database server, ensure that there is enough hard disk space to store the collected image data. You can use a distributed storage system or cloud storage solution to cope with large amounts of image data. For example, in a large scenic spot, several TB of image data may be generated every day, and cloud storage can facilitate data management and expansion.

[0068] The computing resources of the application server, such as CPU and GPU, should be optimized according to the needs of image acquisition and processing. If complex real-time portrait detection algorithms (such as deep learning algorithms) are involved, GPU can be used to accelerate computing to improve the efficiency of image acquisition and detection.

[0069] Select a suitable data transmission protocol between the application server and the camera, such as RTSP (Real-time Streaming Protocol) or HTTP-Live-Streaming (HLS). RTSP is suitable for scenarios with high real-time requirements and can quickly transmit images collected by the camera to the server for detection. At the same time, the bandwidth limitation of the scenic area network should be considered and the image data should be appropriately compressed. For example, the H.264 or H.265 video encoding standard can be used to reduce the amount of data transmission and avoid network congestion while ensuring image quality.

[0070] A dedicated internal network of the scenic spot can be established, or a virtual private network (VPN) can be used to ensure the security and stability of data transmission. Especially for portrait data involving tourists' privacy, a secure transmission network is essential.

[0071] The database server should formulate a reasonable storage strategy for the collected tourist image data. The data can be classified and stored according to time (such as date, time period), location (such as different areas of the scenic spot), etc. For example, the portrait data collected at the entrance of the scenic spot in the morning is stored in a specific folder or database table. Take strict data security measures, such as data encryption and access control. The stored portrait data is encrypted, and only authorized personnel (such as scenic spot security management personnel, public security departments, etc.) can decrypt and access the data under specific circumstances (such as investigating cases, looking for lost tourists, etc.). At the same time, data should be backed up regularly to prevent data loss or damage.

[0072] Furthermore, a red signal light or a signal light of other colors may be provided on the camera device, and after the signal is collected, the light may be turned on to provide feedback to the tourists.

[0073] Beneficial effects of this program:

[0074] 1. It can maintain tourists’ travel experience and costs while increasing their shopping convenience.

[0075] 2. The sales cart can be four-wheeled or track-based. The track-based one is faster and safer.

[0076] 3. Make full use of the camera equipment in the tourist area to detect and collect tourist images in real time. When it is detected that some tourists make certain body movements, it is determined that the tourist has the intention to buy, and the location of the tourist is detected. Then, the sales cart is controlled to go to the location of the tourist to sell goods. This solves the problem that tourists have to take a long route to buy goods, which consumes more physical strength and time. Some tourists even don’t know where the sales point is and need to slowly look for it, etc., which is inconvenient for shopping in tourist areas.

[0077] 4. Compared with calling for takeout, this solution has the following advantages: it can call for the sales device by body movements without using a mobile phone, which is suitable for the elderly, children or people who do not have mobile phones and use cash. Compared with takeout, the delivery time is shorter and the cost is lower.

[0078] Embodiment 2

[0079] like Figure 2 The figure is a flow chart of a control method of an automatic vending system in a tourist area, which is applied to the automatic vending system in a tourist area described in the first embodiment, and includes:

[0080] S1: The camera device detects and collects the tourist image in real time. Further, the camera device detects and collects the tourist image in real time by the high-definition camera of the first embodiment. The following steps are also included:

[0081] Use simple image processing technology to perform preliminary portrait detection. For example, use skin color detection algorithm to locate possible portrait areas. In RGB color space, human skin color usually falls within a specific range. Based on this range, skin color areas in the image can be screened out as candidate areas for portraits. Then, shape analysis (such as detecting whether there are shapes similar to heads and bodies) can be combined to further determine the portrait.

[0082] Motion detection algorithms are used to capture moving portraits. By comparing the differences between consecutive frames, when an obvious moving object is detected and its size and shape match the characteristics of a portrait, it can be determined to be a tourist image. This method is relatively effective in monitoring tourist flow and behavior.

[0083] Use deep learning-based target detection algorithms, such as the YOLO (You Only Look Once) series or Faster-RCNN (Region-CNN). After being trained on large-scale data sets, these algorithms can quickly and accurately detect portraits in images. For example, the YOLOv5 model can divide the input image into multiple grids, each of which is responsible for predicting target objects (including portraits) within a certain range, and output the positions and category probabilities of all detected portraits at one time through the forward propagation of the neural network. Taking advantage of the real-time advantages of deep learning models, portrait detection is performed while collecting images. The model can be deployed on edge computing devices (such as the above-mentioned camera device using a smart camera with an AI chip) or an application server close to the camera to reduce data transmission delays and achieve true real-time detection.

[0084] Furthermore, other image acquisition schemes having the same function may be adopted.

[0085] S2: The system platform analyzes the tourist image to determine the tourist's purchase intention; if the system platform determines that the tourist has a purchase intention, the vending device is controlled to go to the tourist to sell goods.

[0086] In a specific implementation, the system platform analyzes the tourist image to determine the tourist's purchasing intention, including analyzing whether the tourist image has specific body movements. If the tourist makes specific body movements, it is determined that the tourist has purchasing intention.

[0087] Furthermore, analyzing whether the tourist image has specific body movements includes the following steps:

[0088] S211: Estimate human body posture.

[0089] This is a key technology for analyzing tourists' body movements. It mainly estimates the positions of human joints (such as head, shoulders, elbows, wrists, hips, knees, ankles, etc.) through computer vision algorithms. At present, deep learning-based methods perform well in human posture estimation. For example, OpenPose is a popular open source posture estimation model that uses convolutional neural networks (CNNs) to detect key points of the human body. The model generates a heat map on the input tourist image, and the peak position in the heat map corresponds to the position of each joint point. By analyzing the position changes of these joint points, body movements can be inferred.

[0090] Another method is posture estimation based on part affinity field (PAF). It not only determines the location of joint points, but also calculates the degree of association between various limb parts (such as arms and legs), thereby more accurately depicting human posture. For example, when analyzing whether a tourist is waving or letting his arms hang naturally, PAF can help distinguish the connection relationship between joint points in different limb states.

[0091] S212: Extract action features.

[0092] After determining the joints of the human body, it is necessary to extract features that can represent limb movements. A common method is to use joint angle features. For example, calculate the angle between the elbow and shoulder, the angle between the knee and hip, etc. The changes in these angles can reflect the bending and stretching of the limbs. Take the action of a tourist raising his hand as an example. As the arm is raised, the angle between the shoulder and elbow will gradually increase.

[0093] It is also possible to extract the motion trajectory features of the limbs. By tracking the position changes of the joint points in consecutive image frames, the motion path of the limbs can be obtained. For example, when analyzing the walking movements of tourists, observing the motion trajectory of the ankle joint points can determine the walking direction and speed of the tourists.

[0094] S213: Data preprocessing and model training (using machine learning / deep learning methods).

[0095] Collect images or video data containing various body movements of tourists. This data can come from surveillance cameras in actual scenic spots, shots of simulated scenic spots, etc. For each image or video clip, it is necessary to mark the categories of tourists' body movements, such as standing, walking, sitting, waving, jumping, etc. The labeling process can use professional labeling tools, such as LabelImg, etc. The labeler can complete the data labeling by marking the joint positions and action categories on the image.

[0096] In order to improve the generalization ability of the model, the data should be as diverse as possible, including tourists of different ages, body shapes, and clothing, as well as action images in various scenarios (such as sunny, cloudy, indoors, outdoors, etc.).

[0097] S214: Model selection and training.

[0098] Select a suitable model according to the task and data characteristics of the motion analysis. If it is a method based on deep learning, you can choose an architecture such as a convolutional neural network (CNN) or a recurrent neural network (RNN). For motion classification tasks based on joint position, CNN can effectively extract spatial features in the image; and for motion analysis based on motion trajectory, RNN (especially long short-term memory network, LSTM) can process time series data and capture the dynamic changes of the action. In this embodiment, CNN is used to analyze specific actions, such as the action of opening the palm after raising the hand, or other specific body movements that inform tourists in advance, and can effectively extract spatial features in the image.

[0099] Use the labeled data to train the model. Input the image data into the model, and the model performs forward propagation according to the preset network structure and parameters to calculate the predicted action category. Then, by comparing with the labeled true category, calculate the loss function (such as cross entropy loss), and use the back propagation algorithm to update the model parameters to minimize the loss function and improve the accuracy of the model.

[0100] In actual applications in scenic spots, tourists’ images are acquired in real time through cameras. The images are input into the trained motion analysis model, which first estimates the human body posture and obtains the joint positions. Then, motion features such as joint angles and motion trajectories are extracted. Finally, the categories of tourists’ body movements are determined based on these features. For example, when the model detects that the tourist’s arm is raised upwards and the angle between the shoulder and elbow is greater than a certain threshold, it is determined that the tourist is waving.

[0101] S215: In order to achieve real-time performance, the computational efficiency of the model needs to be optimized. Some technical means can be used, such as model quantization (converting model parameters into low-precision data types) and model pruning (removing unimportant connections or neurons in the model), to reduce the computational workload and inference time of the model.

[0102] Furthermore, the specific body movements are displayed to inform tourists when they enter the park or in the recreational area of ​​the tourist area.

[0103] In a specific implementation, the control of the vending device to go to the tourists to sell goods includes the following steps:

[0104] S221: Detect the position and installation angle of the camera device that detects the tourist's purchasing intention.

[0105] Furthermore, if there are several cameras that discover together (i.e., there are overlapping areas of collection between each other), the distance between the tourist and each camera is calculated, and the camera at the closest distance is used as the camera device that discovers the tourist's intention to purchase, so as to prevent the distance from being too far and making it inconvenient for subsequent vehicles to find the tourist; if a camera that is too far away from the tourist is used, the scope of the first area is too large, and the probability of miscalculation will increase during the subsequent second step of approach navigation.

[0106] Furthermore, the following method can be used to calculate the distance between tourists and each camera:

[0107] (1) Using the triangulation method of monocular vision, monocular vision uses only one camera to obtain image information. In the triangulation method, we assume that the optical axis of the camera is parallel to the ground, and we know the height of the camera (set as h) and the angle between the tourist's position in the image and the camera's optical axis (set as θ). According to the trigonometric function relationship, the distance d between the tourist and the camera can be calculated using the tangent function, that is, d = htanθ. Actual operation steps:

[0108] First, you need to determine the height of the camera. This can be done by taking precise measurements when mounting the camera.

[0109] Then, the angle between the tourist's position in the image and the camera's optical axis needs to be calculated. This can be achieved by processing the image. For example, the center of the image can usually be regarded as the position of the camera's optical axis. By determining the pixel distance between the tourist's pixel position in the image and the center of the image, combined with the camera's viewing angle (which can be obtained from the camera's technical parameters), this angle can be calculated.

[0110] (2) Method based on object size and pixel ratio: If we know the actual size of a certain body part of a tourist (such as the average height of a person) (set as H), and can measure the pixel height of this body part in the camera image (set as h p ), and knowing the focal length of the camera (set to f), we can calculate the distance d between the tourist and the camera based on the principle of similar triangles. The formula is Practical steps:

[0111] The focal length of the camera can usually be obtained from the camera's product manual or technical parameters.

[0112] For the actual size of tourists' body parts, such as human height, the average height of adult males is about 175cm, and the average height of adult females is about 160cm. This size can be estimated based on the approximate gender ratio and average height of tourists in the scenic area. If the scenic area has more precise tourist information (such as through height measurement equipment at the entrance), more accurate data can be used.

[0113] Measuring the pixel height of the tourist's body parts in the image can be achieved using image processing software or programming libraries. For example, using the OpenCV library in Python, by processing and analyzing the image, the pixel range of the tourist's body parts (such as head to feet) in the image is determined, thereby obtaining the pixel height.

[0114] (3) Combined with the distance estimation method of deep learning (if there is a deep neural network model), the already trained deep neural network model can be used to estimate the distance between the tourist and the camera. These models are usually trained on a large amount of image data and corresponding distance annotation data. The model will learn the relationship between various features in the image (such as the size of the object, blur, texture changes, etc.) and the actual distance. Actual operation steps:

[0115] First, you need to obtain a suitable deep neural network model. You can use an open source model, or train it yourself on a relevant dataset. For example, some models for monocular depth estimation, such as Monodepth, etc.

[0116] The images of tourists captured by the camera are then fed into the model, which outputs a distance estimate.

[0117] Furthermore, other solutions that can measure the distance between the camera device and the tourist can also be adopted.

[0118] S222: Determine the first area where the tourist is located according to the position and installation angle of the camera device.

[0119] Furthermore, determining the first area where the tourist is located specifically includes the following steps:

[0120] (1) Collect the basic parameters of the camera device.

[0121] Lens focal length: Focal length is one of the key factors that determine the camera range. A shorter focal length (such as a wide-angle lens) can capture a wider area, while a longer focal length (such as a telephoto lens) will narrow the shooting range but can capture objects farther away with clearer details. For example, a lens with a focal length of 2.8mm usually has a very wide angle of view, suitable for shooting large areas, such as the entrance square of a scenic spot; while a lens with a focal length of 12mm has a narrow angle of view, suitable for focusing on specific attractions or crowd areas.

[0122] Horizontal and vertical angle of view: The technical specifications of a camera usually give the horizontal and vertical angle of view in degrees. These angles describe the range that the lens can cover in the horizontal and vertical directions. For example, a camera with a horizontal angle of view of 90° and a vertical angle of view of 45° has a wider range in the horizontal direction and a relatively narrow range in the vertical direction.

[0123] Mounting height: The height of the camera from the ground will also affect its shooting range. A higher mounting height can increase the shooting range, especially when covering a large horizontal area. At the same time, the mounting height will also change the angle between the shooting angle and the ground, thus affecting the shooting effect of objects at different distances.

[0124] (2) Use geometric principles to determine the shooting range.

[0125] Plane geometry model (for horizontal shooting range):

[0126] Assume that the camera is installed horizontally, the center of the lens is point O, and with O as the center of the circle and half of the horizontal viewing angle of the lens (set as α) as the angle, extend rays to both sides. Suppose the installation height of the camera is h, forming a fan-shaped area on the ground with the center directly below the camera. According to the relationship between trigonometric functions, the farthest shooting distance (set as L) can be calculated. For example, if the installation height h = 5 meters and half of the horizontal viewing angle α = 45°, then the farthest shooting distance L = 5×tan45° = 5 meters, and this fan-shaped area is the horizontal shooting range of the camera.

[0127] If the distortion of the lens and the complexity of the actual scene are taken into account, this ideal model needs to be corrected. Lens distortion may cause objects in the edge area to deform, making the shooting range slightly different from the theoretical calculation in actual application. The calculation results can be adjusted through the distortion parameters of the lens (generally provided by the lens manufacturer).

[0128] Three-dimensional geometric model (considering vertical and horizontal extents):

[0129] For the case where both horizontal and vertical shooting ranges are considered, a three-dimensional geometric model is established. Let the horizontal viewing angle of the camera device be α h , the vertical viewing angle is α v , installation height is h. With the camera device as the center, the farthest shooting distance in the horizontal direction The highest shooting point height in the vertical direction Minimum shooting point height (if the lens can shoot downward at a certain angle) (Here, a negative angle means shooting downward.) This determines a three-dimensional cone-shaped shooting range with the camera device as the vertex.

[0130] Combined with the position and installation angle, the position and range of the first area can be obtained.

[0131] In practical applications, the impact of obstructions may also need to be considered. For example, in a scenic area, if the camera is installed next to a building, the building may block part of the shooting range. It is necessary to determine the actual effective shooting area through on-site investigation and geometric calculation based on the actual obstruction situation.

[0132] (3) Use software tools to assist in the determination.

[0133] Many video surveillance software provide camera field of view settings and preview functions. After inputting the focal length, installation height, angle and other parameters of the camera device into the software, the software can automatically draw a graph of the shooting range and display it on a map (if there is a scenic area map integrated) or a virtual scene. This allows you to more intuitively see the relationship between the shooting range and the actual layout of the scenic area.

[0134] For complex scenic environment, 3D modeling and simulation software can be used. First, build a 3D model of the scenic area, including terrain, buildings, vegetation, etc. Then add the camera device to the model according to the actual position and installation angle, and accurately determine the shooting range through the software's ray tracing or field of view simulation function. This method is very useful in the early stages of scenic area planning and security system design, and can help optimize the layout of the camera device to achieve the best monitoring effect.

[0135] Specifically, other methods that can achieve the same effect may also be used.

[0136] S223: planning a path for the vending device to go to the first area according to the location of the vending device, and controlling the vending device to go to and enter the first area; at this time, the vending device and the tourist are both within the first area in the field of view of the camera device.

[0137] S224: Detecting the distance between the vending device and the tourist, and controlling the vending device to move to a preset distance from the tourist according to the distance.

[0138] Further, the detecting the distance between the vending device and the tourist includes detecting the distance between the vending device and the tourist through an image of a camera device, including the following method:

[0139] (1) Geometric method based on monocular vision.

[0140] Similar triangle principle: If the focal length of the camera is known (set as f), and the pixel size of the vending machine and the tourist in the image can be determined (set the height of the vending machine in the image as pixel h1, and the height of the tourist in the image as pixel h2), as well as the relative position relationship between the vending machine and the tourist in the image, and the actual height of the vending machine is also known (set as H), the distance between the two can be calculated using similar triangles. Calculation steps:

[0141] First, according to the proportional relationship of similar triangles, assuming that the actual distance between the tourist and the vending device is d, we can get the formula , where D is a reference distance calculated based on the known actual height H of the vending device, its height h1 in the image, and the focal length f of the camera. Substituting D into the previous formula, we can calculate the value of d. For example, if the focal length of the camera is f = 50 mm, the actual height of the vending device is H = 2 m, the height of the vending device in the image is h1 = 100 pixels, and the height of the visitor is h2 = 50 pixels, we first calculate Then

[0142] (2) Monocular depth estimation method (if there is a pre-trained model).

[0143] Principle: Use the trained monocular depth estimation model, which is usually trained on a large amount of image data with depth labels. The model can estimate the depth information of each pixel based on the image texture, object size and relative position and other features. Calculation steps: Input the image of the vending machine and tourists captured by the camera into the monocular depth estimation model, and the model will output a depth map with a corresponding depth value for each pixel in the map. By extracting the depth values ​​of the vending machine and tourists in the depth map, the distance between them can be calculated.

[0144] (3) Binocular vision system (if the camera is configured as binocular).

[0145] Parallax principle: The binocular vision system consists of two cameras with a certain distance (set as b, the baseline distance). By calculating the parallax of the same object (vending device and tourist) in the two camera images (set as d disparity ), we can use the formula To calculate the distance, f is the focal length of the camera. Calculation steps: First, the binocular camera must be accurately calibrated to obtain the accurate baseline distance b and focal length f. Then, the feature matching algorithm is used to find the pixels corresponding to the vending machine and the visitor in the images of the two cameras and calculate the disparity. For example, if the baseline distance b = 100mm, the focal length f = 50mm, and the disparity pixel d disparity = 10 pixels, then the distance

[0146] (4) Based on depth sensor (if the camera is equipped with one).

[0147] Time of Flight (ToF) Sensors:

[0148] Principle: ToF sensor calculates distance by emitting light pulses and measuring the flight time of the light pulses from emission to return (set as t). According to the formula Where c is the speed of light (about 3×10 8 m / s). Calculation steps: Install a ToF sensor on the camera device and connect it to the application server; when the ToF sensor detects the light pulses reflected by the vending device and the tourists, calculate their distances to the camera respectively. Assuming the distance to the vending device is d1 and the distance to the tourists is d2, the distance between the two is |d1-d2|.

[0149] (5) Structured light sensor:

[0150] Principle: Structured light sensors project specific light patterns (such as stripes, dots, etc.) into the scene, and then calculate the distance based on the deformed light pattern captured by the camera, based on the principle of triangulation.

[0151] Calculation steps: Similar to the ToF sensor, a structured light sensor is installed on the camera device and connected to the application server; the distance between the vending device and the visitor is calculated separately, and then the difference is calculated to obtain the distance between the two.

[0152] Furthermore, the detection of the distance between the vending device and the tourist includes installing a proximity camera on the vending device; the proximity camera is connected to the controller, and the distance between the vending device and the tourist is detected through the image of the proximity camera. The specific steps are the same as the above-mentioned calculation of the distance between the tourist and each camera, or other measurement methods that can achieve the same effect can also be used. For example, a ranging radar is installed on the vending device, the location of the shopping tourist is found through the proximity camera, and the distance between the two is measured by the ranging radar to facilitate approach, thereby reducing the workload of calculation.

[0153] The above-mentioned solution of navigating by image detection within the first area allows the vending device to accurately find the tourists no matter where they go, as long as the tourists are within the first area, which not only facilitates the tourists' shopping but also ensures a sustained travel experience.

[0154] Furthermore, the method of controlling the vending device to go to the tourists to sell goods also includes the following steps:

[0155] S225: Save the image data of tourists who intend to buy goods, and keep relaying detection of the tourists through various cameras in the tourist area to detect the second area where the tourists are in real time, and control the vending device to go to the second area. This solution allows the system to know the location of tourists in real time. Tourists do not have to wait in a fixed place like waiting for takeout. Tourists can wait for the vending cart to find them while playing, so as to maintain the playing experience and shopping experience.

[0156] Furthermore, when a tourist's intention to buy is detected, a drone can be used to fly to the tourist's location, and the tourist can be asked what he wants to buy by voice collection, and then a sales cart can be controlled to go to the tourist's location.

[0157] Furthermore, the specific body movements of tourists can be further detected (for example, the tourist area stipulates in advance what kind of body movements indicate what kind of items they want to buy) to determine the types of items purchased by tourists, and then control the corresponding sales carts to sell the goods, thereby improving the efficiency of the entire purchase and sale.

[0158] Furthermore, a red signal light may be provided on the camera device, and after a good signal is received, the light will be turned on to give feedback to tourists.

[0159] In specific implementation, the voice questions of tourists can also be collected and analyzed using a language analysis model.

[0160] Furthermore, the language analysis model is obtained by fine-tuning the natural language processing basic model according to the scenic spot knowledge, including the following fine-tuning steps:

[0161] (1) Data collection and organization.

[0162] Collect knowledge texts about scenic spots and attractions: First, we need to collect various text materials related to scenic spots and attractions, including introductions to scenic spots, tour guides, tourist comments, historical and cultural backgrounds, etc. For example, for the Palace Museum, we need to collect introductions to palace buildings (such as the architectural style and purpose of the Hall of Supreme Harmony), descriptions of cultural relics and exhibits, and tourists' experience and evaluation of different areas. These texts can be obtained from channels such as official scenic spot websites, travel guide platforms, and materials released by cultural research institutions.

[0163] Clean and annotate data: Clean the collected text and remove noise information, such as advertisements and irrelevant HTML tags. Then annotate according to the specific fine-tuning task. If fine-tuning the scenic spot question and answer, annotate the question type (such as the location of the scenic spot, opening hours, historical events, etc.) and the corresponding answer in the text; if fine-tuning the scenic spot recommendation, annotate the text to reflect the characteristics of the scenic spot and the suitable crowd.

[0164] (2) Select an appropriate fine-tuning strategy.

[0165] Based on pre-trained model architecture: If a model with Transformer architecture (such as BERT, GPT) is used, its pre-trained weights can be used for fine-tuning on the scenic spot knowledge dataset. For BERT, an output layer for a specific task is usually added during the fine-tuning stage based on pre-training. For example, in the scenic spot classification task, a classifier is added after the last layer output of BERT to classify the scenic spots into different categories (such as historical buildings, natural scenery, etc.).

[0166] Multi-task fine-tuning: Consider fine-tuning multiple related tasks at the same time to improve the model's comprehensive understanding of scenic spot knowledge. For example, fine-tune the two tasks of scenic spot question answering and scenic spot recommendation. During the training process, share the underlying parameters of the model, and process the question answering and recommendation tasks separately through different output layers. In this way, the understanding of scenic spot knowledge gained by the model when learning the question answering task can also be used for the recommendation task, and vice versa.

[0167] (3) Fine-tuning process.

[0168] Adjust model parameters: In the fine-tuning stage, use the collected and organized scenic spot knowledge data to update the model parameters at a smaller learning rate. This is because the pre-trained model already has a good language understanding foundation and does not need to change parameters significantly. For example, for a pre-trained LSTM model used for scenic spot name recognition, when fine-tuning, set the learning rate to about 1 / 10 of that in pre-training, and focus on adjusting the output layer and some hidden layer parameters related to the scenic spot name.

[0169] Optimization target setting: Set optimization targets based on specific tasks. If it is a text generation task (such as generating tour guides), the cross entropy loss between the generated text and the reference tour guides can be used as the optimization target to make the content generated by the model as similar as possible to high-quality tour guides. For classification tasks (such as classifying attractions into popular and non-popular), the cross entropy loss function is used in combination with evaluation indicators such as accuracy to optimize the model so that the model can accurately classify attractions.

[0170] (4) Evaluation and verification.

[0171] Internal evaluation indicators: Use indicators such as accuracy, recall, and F1-score to evaluate the performance of the model after fine-tuning. For example, in the tourist information retrieval task, accuracy measures the proportion of correct information retrieved by the model to the total information retrieved, recall measures the proportion of correct information retrieved by the model to all correct information that should be retrieved, and F1-score is a balanced indicator that comprehensively considers accuracy and recall.

[0172] External verification method: Invite professional tour guides, scenic spot staff or experienced tourists to manually verify the output results of the model. For example, for the recommended list of scenic spots generated by the model, let them evaluate the rationality and accuracy of the recommendations and whether they meet the needs of different tourist groups; for the scenic spot questions answered by the model, let them judge whether the answers are accurate, complete and easy to understand.

[0173] (5) Continuous updating and optimization.

[0174] Update data: The knowledge of scenic spots may be updated over time, such as the discovery of new cultural relics, the development of new tourist areas, etc. Therefore, it is necessary to regularly update the fine-tuning data to incorporate new knowledge into the model. For example, when a scenic spot adds a new feature attraction, the introduction, comments and other data of the attraction are added to the fine-tuning dataset in a timely manner.

[0175] Optimize model structure: Optimize the model structure based on feedback from evaluation and actual applications. If you find that the model does not work well when processing long scenic spot introduction texts, you can consider adding a long sequence processing mechanism, such as the Transformer-XL structure; if the model has difficulty understanding and generating knowledge about certain niche scenic spots, you can increase relevant data in a targeted manner and strengthen the training of the corresponding parts of the model.

[0176] The present application provides an automatic vending system for a tourist area, comprising: a vending device, a camera device and a system platform; the camera device is installed in the tourist area and is used to detect the purchase intention of tourists; the camera device is connected to the system platform and sends the data of the detected purchase intention to the system platform; the system platform is connected to the vending device to control the vending device to go to the tourists to sell goods, so as to solve the problem of inconvenience for tourists to buy goods in the tourist area. The camera device in the tourist area is fully utilized to detect and collect the images of tourists in real time. When it is detected that some tourists make certain body movements, it is judged that the tourists have the intention to buy, and the location of the tourists is detected, and then the vending cart is controlled to go to the location of the tourists to sell goods, so as to solve the problem of inconvenience in shopping in the tourist area, such as tourists have to take a long route to buy goods, consume more physical strength and time, and some tourists even do not know where the sales point is and need to slowly look for it.

[0177] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0178] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An automatic vending system in a tourist area, characterized in that: include: A vending device, a camera device and a system platform; the camera device is installed in a tourist area to detect tourists' purchasing intentions; The camera device is connected to the system platform and sends data of detected purchase intention to the system platform; The system platform is connected with the vending device to control the vending device to go to tourists to sell goods, so as to solve the problem of inconvenience for tourists to buy goods in tourist areas.

2. The automatic vending system for tourist areas according to claim 1, characterized in that: The vending device includes an automatic vending cart or a drone.

3. The automatic vending system for tourist areas according to claim 2, characterized in that: The automatic vending cart includes a wheeled type or a tracked type.

4. The automatic vending system for tourist areas according to claim 1, characterized in that: The system platform includes an application server and a database server; the application server is connected to the database server, the vending device and the camera device respectively.

5. The automatic vending system for tourist areas according to claim 1, characterized in that: The camera device adopts a camera; the camera is installed on a pole arranged in the tourist area.

6. A method for controlling an automatic vending system in a tourist area, characterized in that: The automatic vending system applied to a tourist area according to any one of claims 1 to 5 comprises: The camera device detects and collects the images of tourists in real time; The system platform analyzes tourist images to determine their purchasing intentions; If the system platform determines that the tourist has the intention to purchase goods, the vending device is controlled to go to the tourist to sell goods.

7. The method for controlling an automatic vending system in a tourist area according to claim 6, characterized in that: The system platform analyzes the tourist image to determine the tourist's purchasing intention, including analyzing whether the tourist image has specific body movements. If the tourist makes specific body movements, it is determined that the tourist has purchasing intention.

8. The method for controlling an automatic vending system in a tourist area according to claim 7, characterized in that: The specific body movements are notified to tourists by displaying them when they enter the park or in the recreational areas of the tourist area.

9. The method for controlling an automatic vending system in a tourist area according to claim 6, characterized in that: The method of controlling the vending device to go to the tourists to sell goods comprises the following steps: The location and installation angle of the camera device that detects the tourists' purchasing intention; Determine the first area where the tourist is located according to the position and installation angle of the camera device; According to the location of the vending device, a path for the vending device to go to the first area is planned, and the vending device is controlled to go to and enter the first area; The distance between the vending device and the tourist is detected, and the vending device is controlled to move to a preset distance from the tourist according to the distance.

10. The method for controlling an automatic vending system in a tourist area according to claim 9, characterized in that: The control of the vending device to go to the tourists to sell goods also includes the following steps: The image data of tourists with the intention of purchasing goods are saved, and the various camera devices in the tourist area are used to continuously detect the tourist so as to detect the second area where the tourist is located in real time and control the vending device to go to the second area.

Images