Public place intelligent interaction robot control method and interaction robot

Abstract

The invention discloses a public place intelligent interaction robot control method and an interaction robot, and relates to the technical field of artificial intelligence, and the method comprises the steps: taking a user terminal corresponding to an earliest path navigation instruction as a target terminal; sending a shooting instruction to the target terminal, wherein the shooting instruction is used for calling a camera of the target terminal to enable the target terminal to shoot a face image of a target user corresponding to the target terminal; performing semantic recognition on the voice information, determining a first target location where the target user needs to arrive, and performing face recognition on the face image to obtain travel information of the target user; determining a user level of the target user according to the first target location and the travel information; determining an optimal matching robot according to the user level, the first target location, the information of each robot and the first current location; and generating and displaying a path regulation and control scheme of the optimal matching robot. According to the invention, the user experience of using the robot can be effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a control method for an intelligent interactive robot in a public place and an interactive robot. Background Art

[0002] With the rapid development of science and technology, artificial intelligence technology is becoming increasingly mature. Robots, as an important carrier of this technology, are gradually being integrated into people's daily lives, especially in public places. The application of robots has made public places more intelligent, convenient and efficient, bringing a new service experience to the public.

[0003] At present, robots in public places such as airports and shopping malls mainly provide navigation and guide services, helping users to quickly find the target location according to the needs of users, and can also provide guide services. By obtaining the user's target location, the user is taken to the target location according to the generated path. However, in these public places, if the area of ​​the place available for user activities is large and the flow of people is large, there will often be multiple users who need guide services. In this case, since the built-in program of the robot can only serve one user demand and cannot meet all user needs, the user experience is not good. Especially in public places such as airports and stations, when users need time-limited robot navigation services, for example, because the departure time of a flight, train or other means of transportation is approaching, the user needs to reach a certain place within a very limited time, such as a boarding gate, a station ticket gate, etc. At this time, the robot navigation service required by the user needs to be completed within a specific time limit, but because the robot service other users cannot provide corresponding navigation services in time, it may affect the user's subsequent subsequent itinerary arrangements, seriously reducing the user's experience of using robots in public places.

[0004] Therefore, a method is urgently needed to solve the above problems. Summary of the invention

[0005] The embodiments of the present application provide a control method for an intelligent interactive robot in a public place and an interactive robot, which are used to improve the experience of target users in using the robot.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a method for controlling an intelligent interactive robot in a public place is provided, which is applied to a central console, wherein the central console is connected to at least one robot, and each of the robots is preset with a first camera. The method comprises:

[0008] In response to receiving path navigation instructions from multiple user terminals, taking the user terminal corresponding to the path navigation instruction issued earliest as a target terminal;

[0009] In response to receiving the voice information sent by the target terminal, sending a shooting instruction to the target terminal, wherein the shooting instruction is used to call the second camera of the target terminal to enable the target terminal to shoot a facial image of a target user corresponding to the target terminal;

[0010] In response to receiving the face image sent by the target terminal, performing semantic recognition on the voice information to determine a first target location that the target user needs to reach, and performing face recognition on the face image to obtain itinerary information of the target user;

[0011] acquiring robot information of each of the robots in real time, and acquiring a first current location of the target user in real time;

[0012] determining a user level of the target user according to the first target location and the itinerary information;

[0013] Determine the best matching robot according to the user level, the first target location, the information of each robot and the first current location, and obtain the second current location of the best matching robot;

[0014] Based on the first target location, the first current location and the second current location, a path control plan for the optimal matching robot is generated and displayed.

[0015] In a possible implementation manner of the first aspect, generating the path control scheme of the optimal matching robot based on the first target location, the first current location, and the second current location includes:

[0016] generating a first path according to the second current location, the first current location, and the first target location, wherein the first path includes the second current location, the first current location, and the first target location, the second current location is a first waypoint of the first path, the first current location is a second waypoint of the first path, and the first target location is a third waypoint of the first path;

[0017] A first control instruction is generated according to the first path, and the first control instruction is sent to the best matching robot so that the best matching robot executes the first control instruction.

[0018] In a possible implementation manner of the first aspect, the method further includes:

[0019] generating a first collection point according to the second current location and the first target location;

[0020] Generate a second path through the first current location and the first collection point, the second path including the first current location and the first collection point, the first current location being a first waypoint of the second path, and the first collection point being a second waypoint of the second path;

[0021] sending the second path to the target terminal;

[0022] Generate a third path through the first set point and the second current location, the third path including the first set point and the second current location, the second current location being a first waypoint of the third path, and the first set point being a second waypoint of the third path;

[0023] Generate a second control instruction according to the third path, and send the second control instruction to the best matching robot so that the best matching robot executes the second control instruction;

[0024] Generate a fourth path through the first collection point and the first target location, the fourth path including the first collection point and the first target location, the first collection point being a first waypoint of the fourth path, and the first target location being a second waypoint of the fourth path;

[0025] According to the fourth path, a third control instruction is generated, and the third control instruction is sent to the best matching robot, so that the best matching robot executes the third control instruction.

[0026] In a possible implementation manner of the first aspect, before generating a third control instruction according to the fourth path, the method includes:

[0027] determining the arrival time of the target user through the second path;

[0028] Setting a preset waiting time threshold of the optimal matching robot according to the arrival time;

[0029] Within a preset waiting time threshold, controlling the first camera of the optimal matching robot to capture an image;

[0030] Calculating the similarity between the image and the face image, and if the similarity is greater than a preset similarity threshold, the optimal matching robot executes a third control instruction;

[0031] If the waiting time of the best matching robot is greater than the preset waiting time threshold, and the similarity is less than the preset similarity threshold, the best matching robot sends a signal to exit the current service to the central console;

[0032] In response to receiving the signal to exit the current service, a target parking area closest to the best matching robot is determined in a preset parking area, and the parking area position is sent to the best matching robot so that the best matching robot goes to the parking area position.

[0033] In a possible implementation manner of the first aspect, performing semantic recognition on the voice information to determine a first target location that the target user needs to reach includes:

[0034] Converting the voice information into a voice signal;

[0035] Performing endpoint detection on the voice signal to determine the start endpoint and the end endpoint of the voice signal;

[0036] Obtain an audio clip through the start endpoint and the end endpoint;

[0037] Using a preset filtering algorithm on the audio segment to obtain a filtered audio segment;

[0038] Inputting the filtered audio segment into a pre-trained acoustic model to obtain a feature vector of the audio segment;

[0039] Inputting the filtered audio segment into a pre-trained language model to obtain a phrase sequence of the audio segment;

[0040] Performing semantic understanding on the audio clip through the feature vector and the phrase sequence to obtain key information of the audio clip;

[0041] The first target location that the target user needs to reach is determined through the key information.

[0042] In a possible implementation manner of the first aspect, performing endpoint detection on the voice signal to determine a start endpoint and an end endpoint of the voice signal includes:

[0043] Performing frame processing on the speech signal to obtain a plurality of speech frames of the speech signal;

[0044] Performing Fourier transform on each of the speech frames to obtain a spectrogram of each of the speech frames;

[0045] Calculating the spectrum energy value of each of the spectrum graphs;

[0046] When the spectrum energy value is less than a preset lower threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset lower threshold is greater than a preset upper threshold, the first speech frame whose spectrum energy value is less than the preset lower threshold is taken as the starting endpoint of the speech signal;

[0047] When the spectrum energy value is less than a preset high threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset high threshold is greater than the preset high threshold, the last speech frame whose spectrum energy value is greater than the preset high threshold is taken as the end endpoint of the speech signal.

[0048] In a possible implementation manner of the first aspect, determining the user level of the target user according to the first target location and the itinerary information includes:

[0049] Determine the first time node of the target user according to the travel information;

[0050] Determining a path length from the first current location to the first target location according to the first target location and the first current location;

[0051] Get the current time node;

[0052] Calculate the time period for the best matching robot to reach the first target location according to the path length and the preset speed of the best matching robot, and calculate the second time node for the best matching robot to reach the first target location according to the current time node and the time period;

[0053] Mapping the first path into a preset geographic data model, determining influencing factors of the first path, and determining a third time node for the optimal matching robot to arrive at the first target location according to the influencing factors;

[0054] Using a grey correlation analysis algorithm, constructing an evaluation matrix for the first time node, the second time node, and the third time node;

[0055] Determine the first time node, the second time node, and a weight value of the second time node;

[0056] The weight value and the evaluation matrix are synthesized by using a preset synthesis operator to obtain a comprehensive evaluation result of the user level;

[0057] The user level of the target user is determined based on the comprehensive evaluation result of the user level.

[0058] In a possible implementation manner of the first aspect, the robot information includes robot device information, and determining the best matching robot according to the user level, the first target location, and the robot information includes:

[0059] Acquire spatial location data of the robot;

[0060] Mapping the target user, the first target location, and the robot into the geographic data model and displaying them;

[0061] According to the user level, determining the time threshold of the user in a preset user level database, the time threshold comprising a first time period and a second time period, the first time period being a time period between a first current location and a first target location, and the second time period being a time period between a second current location and the first current location;

[0062] Retrieving a target robot in the geographic data model according to the time threshold;

[0063] Determining environmental information of the public place through the geographic data model;

[0064] Determine in real time the time and power consumption required for the target robot to reach the first target location via the location of the target user through the environmental information of the public place;

[0065] Determining the power reserve of the target robot through the robot device information of the target robot;

[0066] When the power reserve of the target robot is greater than the pre-consumed power and the time required for the target robot to reach the first target location via the first current location is less than a preset time threshold, the optimal matching robot is determined.

[0067] In a possible implementation manner of the first aspect, after determining a first target location that the target user needs to reach and performing face recognition on the face image to obtain the itinerary information of the target user, the method further includes:

[0068] Retrieving a second target location closest to the target user in the geographic data model;

[0069] Obtaining the destination of the target user according to the itinerary information of the target user, and determining whether the first target location and the second target location are the destination;

[0070] If the first target location and the second target location are both the destination, generating second target location confirmation information to the target terminal according to the second target location;

[0071] After receiving the confirmation information sent by the target terminal, taking the second target location as the first target location;

[0072] After receiving the negative information sent by the target terminal, the steps of acquiring the robot information of each robot in real time and acquiring the first current location of the target user in real time are performed.

[0073] In a second aspect, the present application provides an interactive robot, which is applied to a robot, each of the interactive robots is connected to a central console, and each of the interactive robots is preset with a first camera, and the interactive robot further includes:

[0074] a memory configured to store instructions; and

[0075] The processor is configured to call the instructions from the memory and execute them.

[0076] Through the above technical solution, the path navigation instructions of multiple user terminals are received, and the user terminal corresponding to the earliest path navigation instruction is used as the target terminal, which can effectively avoid the problem of long waiting time for users and improve service efficiency. Identity authentication through facial images not only improves security, but also increases the convenience of user interaction with robots, simplifies the use process, and improves the user experience. After receiving the facial image sent by the target terminal, the voice information is semantically recognized to determine the first target location that the target user needs to reach. Through semantic recognition technology, the user's instructions can be accurately identified, and the user's needs and intentions can be better understood, providing personalized services for users and improving user experience. By obtaining the user's itinerary information and determining the user's urgency according to the user level, better target users can be provided with more complete services to ensure that the target user's subsequent itinerary arrangements are not affected. By matching the optimal robot for the target user, a faster path planning can be provided for the target user. According to the user's positioning and the robot's positioning, the path control plan is determined, which can provide users with clear and accurate navigation guidance and improve user satisfaction.

[0077] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of a method for controlling an intelligent interactive robot in a public place provided in an embodiment of the present application;

[0079] Figure 2 A schematic diagram of a process for determining a target location provided in an embodiment of the present application;

[0080] Figure 3 A schematic diagram of a flow chart of another method for controlling an intelligent interactive robot in a public place provided in an embodiment of the present application;

[0081] Figure 4A structural diagram of a first collection point confirmation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0083] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0084] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0085] Figure 1 The following is a schematic diagram showing a flow chart of a method for controlling an intelligent interactive robot in a public place according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a method for controlling an intelligent interactive robot in a public place, which may include the following steps.

[0086] S110, in response to receiving path navigation instructions from multiple user terminals, taking the user terminal corresponding to the path navigation instruction issued earliest as a target terminal;

[0087] S120, in response to receiving the voice information sent by the target terminal, sending a shooting instruction to the target terminal, where the shooting instruction is used to call a second camera of the target terminal to enable the target terminal to shoot a facial image of a target user corresponding to the target terminal;

[0088] S130, in response to receiving the face image sent by the target terminal, performing semantic recognition on the voice information, determining the first target location that the target user needs to reach, and performing face recognition on the face image to obtain itinerary information of the target user;

[0089] S140, acquiring robot information of each robot in real time, and acquiring a first current location of a target user in real time;

[0090] S150, determining a user level of the target user according to the first target location and the itinerary information;

[0091] S160, determining the best matching robot according to the user level, the first target location, the information of each robot and the first current location, and obtaining the second current location of the best matching robot;

[0092] S170: Based on the first target location, the first current location, and the second current location, generate and display a path control plan that best matches the robot.

[0093] Figure 3 The following is a schematic diagram showing another method for controlling an intelligent interactive robot in a public place according to an embodiment of the present application. Figure 3 As shown, the flowchart of the control method of the intelligent interactive robot in public places includes a target terminal 10, a central console 20 and a robot 30. The central console sends verification information to the target terminal. When the target user agrees to perform face recognition verification, the central console obtains the itinerary information and the first target location of the target user, determines the user level, and sends the user level to the robot.

[0094] In response to receiving the path navigation instructions from multiple user terminals, the user terminal corresponding to the earliest issued path navigation instruction is used as the target terminal. In this embodiment, the path navigation instruction refers to an instruction to send a target site to the robot, and the robot guides the user to the target site according to a pre-set route. The target terminal refers to various devices used by the user. In this embodiment, the target terminal can be a mobile phone, a tablet computer and other devices. That is to say, after receiving the path navigation instructions from multiple user terminals, such as mobile phones, the mobile phone that issued the path navigation instruction the earliest is used as the target terminal.

[0095] After determining the target terminal, when a voice message sent by the target terminal is received, a shooting command is sent to the target terminal, and the target user using the target terminal is photographed through the second camera of the target terminal to obtain a facial image of the target user corresponding to the target terminal. In this embodiment, the shooting command is used to control the second camera of the target terminal to capture the facial image of the target user.

[0096] After receiving the face image of the user sent by the target terminal, the first target location that the target user needs to reach is determined by semantically recognizing the user's voice information. In this embodiment, the first target location is the target location determined by recognizing the user's voice information. The semantic recognition of the user's voice information can be achieved through a hidden Markov model. The hidden Markov model is a probability model about time series, which can be used to model voice signals and recognize words or phonemes in voice by learning probability distribution. The hidden Markov model can be used to determine the keywords in the user's voice information, and the first target location that the target user needs to reach is determined through the keywords. After receiving the face image of the user, face recognition is performed on the face image. Face recognition on the face image is a technology for identity recognition based on human facial feature information. It uses computer vision and pattern recognition technologies to detect, extract features and compare faces in face images or videos, thereby realizing automatic verification and recognition of personal identity. When the user agrees to face recognition and passes the face recognition verification, the itinerary information of the target user is obtained.

[0097] After obtaining the itinerary information of the target user, the robot information of each robot is obtained in real time. Various information of the robot can be collected in real time by equipping the robot with various sensors, such as visual sensors, force sensors, sound sensors and temperature sensors. The visual sensor can be used to monitor the environment around the robot in real time, as well as to identify and track objects; the force sensor can measure the force between the robot and other objects such as humans, other robots or objects in the working environment; the sound sensor can monitor the sound in the environment, which helps to understand the dynamic situation around the robot; the temperature sensor can monitor the temperature of the environment to help the robot adapt to different working environments. While obtaining the robot information of each robot in real time, the first current location of the target user can be obtained in real time. The first current location of the target user can be determined by determining the location of the target terminal of the target user. The location of the target terminal of the target user can be determined by GPS positioning. GPS positioning is a global radio satellite navigation system. For example, if the target terminal of the target user is a mobile phone, its built-in GPS module can be used to obtain accurate mobile phone location information to determine the first current location of the target user.

[0098] According to the first target location and the itinerary information, the user level of the target user is determined. In this embodiment, the user level is determined by analyzing the time urgency of the target user from the first current location to the first target location. The itinerary information refers to the detailed information related to the target user's transportation. That is to say, the first time node of the target user is determined through the itinerary information. In this embodiment, the first time node refers to the latest time of the target user's transportation tool. For example, if the target user's transportation tool is a train, the first time node is the latest time node of taking the train. After determining the first time node of the target user, the time from the first target location to the first current location is determined, and this time is used as the second time node. Subsequently, the time period between the optimal matching robot arriving at the first target location and the delayed time period caused by environmental factors in this process to the optimal matching robot arriving at the first target location are used as the third time node. Through the gray correlation analysis algorithm, the first time node, the second time node and the third time node are constructed into an evaluation matrix. The gray correlation analysis algorithm is a statistical analysis method based on the gray system theory, which is mainly used to evaluate the correlation between multiple systems or variables. The weight values ​​of the first time node, the second time node and the second time node are determined by using the hierarchical analysis method. The hierarchical analysis method is a decision analysis tool that constructs a hierarchical structure model, performs pairwise comparisons, and calculates weight values. Next, the weight values ​​are synthesized with the evaluation matrix using a preset synthesis operator to obtain a comprehensive evaluation result of the user level. In this embodiment, the preset synthesis operator can be a weighted average method. The weighted average method is a mathematical method used to calculate the average value of a set of data while considering the importance or weight of each data point. The user level of the target user is determined through the comprehensive evaluation result of the user level.

[0099] After determining the user level of the target user, the best matching robot is determined according to the user level, the first target location, the information of each robot and the first current location. That is to say, when determining the best matching robot, it is necessary to comprehensively consider multiple factors such as the user level, the first target location, and the specific information of each robot, such as the current power reserve of the robot. Specifically, the latest travel time node of the target user is determined through the user level. Subsequently, the time required for the robot to reach the first target location after reaching the location of the target user and the power required to be consumed are determined. The robot is searched at the location of the target user, and the robot whose power reserve is greater than the power required to be consumed and whose time required for the robot to reach the first target location after reaching the location of the target user is less than the latest travel time node of the target user is selected as the best matching robot. After determining the best matching robot, the second current location of the best matching robot is obtained. In this embodiment, the second current location is the location information of the best matching robot. The location of the robot is determined by using wireless local area network technology by measuring the received WiFi signal strength or phase information and comparing it with the WiFi access point at a known location.

[0100] Based on the first target location, the first current location and the second current location, the path control scheme of the best matching robot is generated and displayed. That is to say, the path control scheme of the best matching robot is generated and displayed to the target terminal of the target user through the location of the target user, the location of the best matching robot and the first target location. Specifically, different path control schemes can be generated according to the location of the target user, the location of the best matching robot and the first target location. For example, when the best matching robot is in the middle of the target user and the first target location, a first collection point can be generated. After the target user and the best matching robot arrive at the first collection point, they go to the first target location together. In addition, when the target user is in the middle of the best matching robot and the first target location, the best matching robot can arrive at the location of the target user and then go to the first target location together. After the path control scheme of the best matching robot is generated, the path control scheme is displayed to the target terminal of the target user. The path control scheme can be integrated into the application by a mobile application, such as developing a mobile application. The user can download and install the application to view the real-time location and path planning of the robot on the mobile device. The mobile application can support real-time updates and push notifications so that the user can obtain the latest path control information in time.

[0101] Through the above technical solution, the path navigation instructions of multiple user terminals are received, and the user terminal corresponding to the earliest path navigation instruction is used as the target terminal, which can effectively avoid the problem of long waiting time for users and improve service efficiency. Identity authentication through facial images not only improves security, but also increases the convenience of user interaction with robots, simplifies the use process, and improves the user experience. After receiving the facial image sent by the target terminal, the voice information is semantically recognized to determine the first target location that the target user needs to reach. Through semantic recognition technology, the user's instructions can be accurately identified, and the user's needs and intentions can be better understood, providing personalized services for users and improving user experience. By obtaining the user's itinerary information and determining the user's urgency according to the user level, better target users can be provided with more complete services to ensure that the target user's subsequent itinerary arrangements are not affected. By matching the optimal robot for the target user, a faster path planning can be provided for the target user. According to the user's positioning and the robot's positioning, the path control plan is determined, which can provide users with clear and accurate navigation guidance and improve user satisfaction.

[0102] In one implementation of this embodiment, based on the first target location, the first current location, and the second current location, generating a path control scheme that best matches the robot includes:

[0103] S210, generating a first path according to the second current location, the first current location and the first target location, wherein the first path includes the second current location, the first current location and the first target location, the second current location is a first waypoint of the first path, the first current location is a second waypoint of the first path, and the first target location is a third waypoint of the first path;

[0104] S220: Generate a first control instruction according to the first path, and send the first control instruction to the best matching robot so that the best matching robot executes the first control instruction.

[0105] Based on the first target location, the first current location and the second current location, a path control scheme of the optimal matching robot is generated. In this embodiment, the first target location is the location that the target user needs to reach, the first current location is the location of the target user, and the second current location is the location of the optimal matching robot. Specifically, first, a first path is generated based on the second current location, the first current location and the first target location. In this embodiment, the first path is the path from the location of the optimal matching robot to the location of the target user to reach the first target location, wherein the second current location is the first waypoint of the first path, the first current location is the second waypoint of the first path, and the first target location is the third waypoint of the first path. In other words, when the first current location is between the first target location and the second current location, at this time, the distance from the location of the optimal matching robot to the first target location is farther than the distance from the target user to the first target location, and the target user needs to wait in situ for the optimal matching robot to arrive at the location of the target user before going to the first target location together.

[0106] After obtaining the first path, a first control instruction is generated according to the first path. In the present embodiment, the first control instruction is to make the best matching robot move along the first path. That is to say, after the first path is generated, the first control instruction is issued to the best matching robot, so that the best matching robot moves along the first path, first from the location of the best matching robot, to the location of the target user, and finally to the first target location.

[0107] By generating a first path through the second current position, the first current position and the first target location, and sending a first control instruction to the optimal matching robot, the robot's movement time and energy consumption can be reduced, the robot's work efficiency and accuracy can be significantly improved, and the user's experience of interacting with the robot can be provided.

[0108] In one implementation of this embodiment, the method further includes:

[0109] S310, generating a first collection point according to the second current location and the first target location;

[0110] S320, generating a second path through the first current location and the first collection point, where the second path includes the first current location and the first collection point, the first current location is a first waypoint of the second path, and the first collection point is a second waypoint of the second path;

[0111] S330, sending the second path to the target terminal;

[0112] S340, generating a third path through the first set point and the second current location, the third path including the first set point and the second current location, the second current location being a first waypoint of the third path, and the first set point being a second waypoint of the third path;

[0113] S350, generating a second control instruction according to the third path, and sending the second control instruction to the best matching robot, so that the best matching robot executes the second control instruction;

[0114] S360, generating a fourth path through the first assembly point and the first target location, the fourth path including the first assembly point and the first target location, the first assembly point being a first waypoint of the fourth path, and the first target location being a second waypoint of the fourth path;

[0115] S370. Generate a third control instruction according to the fourth path, and send the third control instruction to the best matching robot so that the best matching robot executes the third control instruction.

[0116] According to the second current location and the first target location, a first gathering point is generated. In this embodiment, the first gathering point is the location where the target user and the best matching robot gather or concentrate. That is to say, a location where the target user and the best matching robot meet is generated between the location of the best matching robot and the first target location. When the first target location is between the second current location and the location of the target user, the target user and the best matching robot go to the first gathering point at the same time, and then reach the first target location together, which can effectively save the target user's time. Figure 4 As shown, the embodiment of the present application provides a structural schematic diagram of determining a first set point, such as Figure 4 As shown, point A is the location of the best matching robot, point B is the location of the target user, point C is the first target location, and D is the first collection point. When the target user is in the middle between the location of the best matching robot, the first collection point is generated in the middle between the location of the target user and the location of the best matching robot, which reduces the time it takes for the target user to reach the first target location.

[0117] The second path is generated through the first current location and the first collection point. The Dijkstra algorithm can be used. The Dijkstra algorithm is an algorithm for finding the shortest path from a single source in a weighted graph. The second path is suitable for finding the shortest path from a single source in a weighted graph. In this embodiment, the second path is the path for the target user to reach the first collection point, wherein the first current location is the first waypoint of the second path, and the first collection point is the second waypoint of the second path. That is to say, after the first collection point is determined, the second path is generated through the location of the target user and the first collection point, and the generated second path is sent to the target terminal of the target user. The second path is the path that guides the user to the first collection point.

[0118] The second path can be sent to the target terminal through network transmission. Network transmission refers to the process of transmitting data from a sender to a receiver in a computer network. The second path can be transmitted through a network protocol. The sender encapsulates the second path into a data packet and sends it to the target terminal through the network. After receiving the data packet, the target terminal decapsulates it to obtain the second path and sends the second path to the target terminal. The target user can reach the first collection point according to the second path in the target terminal.

[0119] A third path is generated through the first gathering point and the second current location. In this embodiment, the third path is the path from the second current location of the best matching robot to the first gathering point, wherein the second current location is the first waypoint of the third path, and the first gathering point is the second waypoint of the third path. That is to say, the path from the best matching robot to the first gathering point is determined through the second current location and the first gathering point where the best matching robot is currently located, so that the best matching robot and the target user arrive at the first target location together after meeting at the first gathering point.

[0120] According to the third path, a second control instruction is generated. In the present embodiment, the second control instruction is used to instruct the best matching robot to move along the third path. That is to say, after the third path is determined, a second control instruction is issued to the best matching robot to make the best matching robot reach the first assembly point and then guide the target user to the first target location.

[0121] After issuing the second control instruction to the best matching robot, a fourth path is generated through the first assembly point and the first target location. In this embodiment, the fourth path refers to the path from the first assembly point to the first target location, wherein the first assembly point is the first waypoint of the fourth path, and the first target location is the second waypoint of the fourth path. That is to say, after the best matching robot and the target user meet at the first assembly point, the best matching robot guides the target user to the first target location.

[0122] After the fourth path is determined, a third control instruction is generated according to the fourth path. In this embodiment, the third control instruction is used to instruct the best matching robot to move along the fourth path. That is to say, after the fourth path is determined, the third control instruction is issued to the best matching robot, so that after the best matching robot reaches the first assembly point, it guides the target user to the first target location.

[0123] By generating a path control plan for the best matching robot through the first target location, the first current location and the second current location, the best matching robot can reach the destination more quickly and accurately, thereby improving the overall task execution efficiency and improving the target user's experience in using the robot.

[0124] In one implementation manner of this embodiment, before generating the third control instruction according to the fourth path, the method includes:

[0125] S410, determining the arrival time of the target user through the second path;

[0126] S420, setting a preset waiting time threshold of the optimal matching robot according to the arrival time;

[0127] S430, controlling the first camera of the optimal matching robot to capture an image within a preset waiting time threshold;

[0128] S440, calculating the similarity between the image and the face image, and if the similarity is greater than a preset similarity threshold, the best matching robot executes the third control instruction;

[0129] S450: If the waiting time of the best matching robot is greater than the preset waiting time threshold and the similarity is less than the preset similarity threshold, the best matching robot sends a signal to exit the current service to the central console;

[0130] S460. In response to receiving a signal to exit the current service, determine a target parking area closest to the best matching robot in the preset parking area, and send the parking area position to the best matching robot so that the best matching robot goes to the parking area position.

[0131] Before generating the third control instruction according to the fourth path, it is necessary to determine whether the target user has arrived at the first collection point. When the target user arrives at the first collection point, the optimal matching robot executes the third control instruction and takes the target user to the first target location. When the target user does not arrive at the first collection point, the optimal matching robot sends a signal to exit the current service to the central console, and then the optimal matching robot returns to the parking area. Specifically, first, the arrival time of the target user is determined through the second path. In this embodiment, the second path is the path of the target user from the first current location to the first collection point. In other words, the time of the target user from the first current location to the first collection point can be determined by obtaining the path length from the first current location to the first collection point and the average walking speed of the user. The average walking speed of the user can be obtained through a preset database. The time of the target user from the first current location to the first collection point can be obtained by dividing the path length by the average walking speed of the user.

[0132] After obtaining the arrival time of the target user, the preset waiting time threshold of the optimal matching robot is set according to the arrival time. In the present embodiment, the preset waiting time threshold is to use the time period from the first current location of the target user to the first assembly point as the preset waiting time threshold. That is to say, after determining the time period from the first current location of the target user to the first assembly point, this time period is used as the preset waiting time threshold of the optimal matching robot. The optimal matching robot will arrive at the first assembly point in advance and wait for the target user at the first assembly point. The waiting time for the target user is the preset waiting time threshold.

[0133] When the best matching robot is waiting for the target user, the first camera of the best matching robot is controlled to capture images. That is to say, within the preset waiting time threshold, the best matching robot will control the first camera of the best matching robot to capture images while waiting for the target user, and store the captured images in the built-in memory of the best matching robot. Subsequently, the images stored in the memory by the best matching robot are extracted for comparison to determine whether the target user has arrived.

[0134] After the first camera of the optimal matching robot is controlled to capture an image, the image and the face image are similarly calculated. The Euclidean distance between the image and the face image can be calculated. The Euclidean distance is a straight-line distance between two points in an n-dimensional space, or the natural length between two points. It is one of the most common distance measurement methods and is usually used to calculate the similarity or difference between vectors. By calculating the straight-line distance between each pixel in the image and the face image, that is, the square root of the sum of the squares of the difference of each pixel in the image, the pixel difference between the image and the face image can be compared to determine whether the target user has arrived at the first collection point. When the similarity is greater than the preset similarity threshold, the preset similarity threshold in this embodiment can be determined according to actual conditions, that is, the smaller the distance between the pixel points of the image and the face image, the more similar it is, that is, the optimal matching robot captures the image of the target user, indicating that the target user has arrived at the first collection point. After determining that the target user has arrived at the first collection point, the optimal matching robot executes the third control instruction, that is, the optimal matching robot will take the target user to the first target location.

[0135] If the waiting time of the best matching robot is greater than the preset waiting time threshold, and the similarity is less than the preset similarity threshold, the best matching robot sends a signal to exit the current service to the central console. That is to say, when the best matching robot waits for the target user at the first collection point for more than the preset waiting time threshold, it means that the target user did not arrive at the first collection point on time within the preset waiting time threshold, and the similarity between the captured image and the target user's facial image is less than the preset similarity threshold, that is to say, the best matching robot did not capture and capture the target user's facial image, indicating that the target user has not currently arrived at the first collection point. At this time, it means that the target user has not arrived at the first collection point. After determining that the target user has not arrived at the first collection point, the best matching robot will send a signal to exit the current service to the central console, and the best matching robot will exit the service process of leading the target user.

[0136] In response to receiving a signal to exit the current service, the target parking area closest to the best matching robot is determined in the preset parking area, and the parking area position is sent to the best matching robot so that the best matching robot goes to the parking area position. That is to say, when the central control console receives the signal to exit the current service, it will determine the position of the target parking area closest to the best matching robot. In this embodiment, the target parking area is a designated area where robots that do not perform leading services are parked. After determining the position of the target parking area closest to the best matching robot, the best matching robot goes to the parking area position, waits for the next target user, and provides the leading service process.

[0137] By determining whether the target user has arrived at the first gathering point before the optimal matching robot's third control instruction, the robot can be effectively prevented from waiting in vain for a long time and can be reallocated to other tasks, thereby improving the robot's utilization rate. When the robot exits the current service, the system will find the nearest target parking area for the robot based on the preset parking area and send the location information. This not only ensures the orderly parking of the robot, but also reduces the time the robot spends looking for a parking location, thereby improving the overall system's operating efficiency.

[0138] In one implementation of this embodiment, semantic recognition is performed on the voice information to determine the first target location that the target user needs to reach, including:

[0139] S510, converting the voice information into a voice signal;

[0140] S520, performing endpoint detection on the voice signal to determine the start endpoint and the end endpoint of the voice signal;

[0141] S530, obtaining an audio clip through the start endpoint and the end endpoint;

[0142] S540, using a preset filtering algorithm on the audio segment to obtain a filtered audio segment;

[0143] S550, inputting the filtered audio segment into a pre-trained acoustic model to obtain a feature vector of the audio segment;

[0144] S560, inputting the filtered audio segment into a pre-trained language model to obtain a phrase sequence of the audio segment;

[0145] S570, performing semantic understanding on the audio clip through the feature vector and the phrase sequence to obtain key information of the audio clip;

[0146] S580: Determine the first target location that the target user needs to reach based on the key information.

[0147] Perform semantic recognition on voice information to determine the first target location that the target user needs to reach. Semantic recognition is based on artificial intelligence technology, which enables computers to understand and analyze the meaning and semantics of natural language text or speech, and convert the meaning in the text into a computer-readable form. Specifically, first, the voice information is converted into a voice signal. The sound waves of the target user can be collected through the microphone on the target terminal, and the collected sound waves are converted into electrical signals to obtain a voice signal.

[0148] After obtaining the speech signal, endpoint detection is performed on the speech signal to determine the start and end endpoints of the speech signal. The double threshold endpoint detection method based on short-time energy and short-time average zero-crossing rate can be used. Short-time energy means that the speech signal can be considered to be steady and unchanged within the "short time" range. Therefore, the speech signal can be divided into different frames to analyze its characteristic parameters. When there is speech, the energy is large, and when there is no speech, the energy is small. The short-time average zero-crossing rate indicates the number of times a frame of speech signal waveform crosses the zero level of the horizontal axis. The average zero-crossing rate of the voiced segment is low and concentrated in the low frequency band, and the average zero-crossing rate of the unvoiced segment is high and concentrated in the high frequency band. The short-time average zero-crossing rate can be used to find the speech signal from the background noise and determine the starting and ending positions of the non-speech segment and the speech segment. Endpoint detection is a key step in speech signal processing. It is mainly used to identify the speech and non-speech areas in the speech signal and accurately locate the start and end points of the speech. In this embodiment, endpoint detection of speech signals is performed based on a threshold-based endpoint detection method, which extracts time domain features such as short-time energy, short-term zero-crossing rate, or frequency domain features such as Mel-frequency cepstral coefficients MFCC, spectral entropy, and sets reasonable thresholds to achieve the purpose of distinguishing speech from non-speech.

[0149] After determining the start endpoint and the end endpoint of the speech signal, the audio segment is obtained through the start endpoint and the end endpoint. That is, by determining the start endpoint and the end endpoint of the speech signal, the audio segment containing effective speech content can be extracted from the original audio.

[0150] The audio clip is subjected to a preset filtering algorithm to obtain a filtered audio clip. The preset filtering algorithm may be a filtering method based on fast Fourier transform, which converts the time domain signal to the frequency domain through the FFT algorithm, and performs filtering processing on the signal in the frequency domain, such as low-pass, high-pass, and band-pass filtering, and then performs inverse FFT on the filtered frequency domain signal to restore it back to the time domain. Filtering the audio clip can improve the audio quality and remove noise. The noise in public places is generally the sound of the air conditioner outdoor unit running, crowd noise, etc. The preset filtering algorithm can reduce the impact of noise.

[0151] The filtered audio clip is input into the pre-trained acoustic model to obtain the feature vector of the audio clip. In this embodiment, the feature vector of the audio clip includes features such as pitch, volume, timbre, and rhythm. The feature vector is extracted from the audio signal, which can reflect the important attributes of the audio content. The pre-trained acoustic model can be a convolutional neural network. The convolutional neural network is one of the representative algorithms in deep learning and belongs to a feedforward neural network. It has a deep structure and convolution calculation characteristics. It extracts local features of the input data through convolution operations to form a complex feature representation, which is ultimately used for tasks such as classification and regression. Specifically, the filtered audio clip is input into the pre-trained acoustic model, and the feature vector of the audio clip is output. The feature vector usually contains information about the audio content, such as pitch, rhythm, timbre, and other information.

[0152] After obtaining the feature vector of the audio clip, the filtered audio clip is input into the pre-trained language model to obtain the phrase sequence of the audio clip. The phrase sequence of the audio clip refers to converting an audio signal into a series of phrases arranged in chronological order after being processed by speech recognition technology. The pre-trained language model can be a neural network model, which is used to capture the relationship between words and improve the performance of the language model. In other words, the acoustic model is responsible for converting the audio signal into a series of acoustic features such as phonemes, and the language model is used to evaluate the rationality of sentences or phrases composed of these modeling units.

[0153] Through the feature vector and phrase sequence, the audio clip is semantically understood to obtain the key information of the audio clip. In this embodiment, the key information of the audio clip includes location information and time information. The voice signal is converted into a pronunciation sequence through the acoustic model. The acoustic model can recognize the phonemes and syllables in the voice signal by learning the feature vector. According to the results of the acoustic model, the word sequence with the highest probability of occurrence is obtained. The language model can infer the phrase sequence that best conforms to grammatical rules and semantic logic by counting the relationship between words in the text. Through the word sequence with the highest probability of occurrence, the key information of the audio clip of the target user is obtained, such as the location that the target user needs to reach within a time period.

[0154] Through the key information, the first target location that the target user needs to reach is determined. That is, through the pre-trained acoustic model and the pre-trained language model, the key information in the audio clip of the target user is obtained, and the key information is parsed. For example, the word sequence with the highest frequency of appearance in the key information is determined to determine the first target location that the target user needs to reach. In this embodiment, the first target location is the location that the target user needs to reach.

[0155] By accurately determining the start and end points of the voice signal, the silence and noise parts can be effectively removed, and the audio segments containing effective voice information can be retained, which helps to reduce the complexity of subsequent processing and improve the accuracy of recognition. Through efficient voice processing and semantic recognition algorithms, the system can quickly parse the voice information input by the user and determine the target location. This helps to improve the user experience and enable users to quickly obtain the information or services they need.

[0156] In one implementation of this embodiment, endpoint detection is performed on a speech signal to determine a start endpoint and an end endpoint of the speech signal, including:

[0157] S610, performing frame processing on the voice signal to obtain a plurality of voice frames of the voice signal;

[0158] S620, performing Fourier transform on each speech frame to obtain a spectrogram of each speech frame;

[0159] S630, calculating the spectrum energy value of each spectrum graph;

[0160] S640: when the spectrum energy value is less than a preset lower threshold and the spectrum energy value of a speech frame adjacent to a speech frame having a spectrum energy value less than the preset lower threshold is greater than a preset upper threshold, the first speech frame having a spectrum energy value less than the preset lower threshold is taken as the starting endpoint of the speech signal;

[0161] S650: When the spectrum energy value is less than a preset high threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset high threshold is greater than the preset high threshold, the last speech frame whose spectrum energy value is greater than the preset high threshold is taken as the end point of the speech signal.

[0162] Endpoint detection is performed on the speech signal to determine the start endpoint and the end endpoint of the speech signal. Specifically, first, the speech signal is framed to obtain multiple speech frames of the speech signal. Before the speech signal is framed, it is usually necessary to perform windowing on the speech signal to reduce spectrum leakage and truncation effects. According to the determined frame length, the windowed speech signal is framed, which usually involves dividing the speech signal into multiple short segments with a length of the frame length, and there is a certain overlap between two adjacent frames. The overlap between two adjacent frames is subjected to frame shifting. Frame shift refers to the length of the overlap between two adjacent frames during the speech signal framing process. It is an important parameter in speech signal processing and is used to ensure the continuity and smoothness of the speech signal. By framing the speech signal, the continuity of the balanced speech signal is guaranteed.

[0163] Secondly, perform Fourier transform on each speech frame to obtain the spectrogram of each speech frame. Performing Fourier transform on each speech frame is a key step in speech signal processing. It can convert the speech signal in the time domain to the frequency domain, thereby obtaining the spectrogram of each speech frame. The spectrogram shows the energy distribution of the speech signal at different frequencies. Applying Fourier transform to each speech frame and converting it from the time domain to the frequency domain will result in a complex array, in which each element corresponds to a frequency component, and its amplitude represents the energy size on the frequency component, and the phase represents the phase information of the frequency component. The square of the energy or amplitude on each frequency component is calculated, and these energy values ​​are plotted into a graph to obtain the spectrogram of each speech frame.

[0164] After obtaining the spectrum of each speech frame, the spectrum energy value of each spectrum is calculated. The spectrum energy value is a concept used to describe the energy size of the signal distributed on different frequency components. By calculating the spectrum energy value of each spectrum, the start and end points of the speech signal can be determined. The spectrum energy value of each spectrum can be determined by calculating the spectrum entropy value in each spectrum. The spectrum entropy value is a concept in information theory, which describes the uncertainty or complexity of the energy distribution of the signal in the frequency domain. The higher the spectrum entropy value, the more uniform the energy distribution of the signal in the frequency domain, that is, the smaller the energy difference of the signal in each frequency component; conversely, the lower the spectrum entropy value, the more uneven the energy distribution of the signal in the frequency domain, and there are obvious frequency components with concentrated energy.

[0165] The spectral entropy value of each spectrum graph can be calculated by the spectral entropy value calculation formula, which is as follows:

[0166] S=-∑p_i*log_2(p_i)

[0167] Among them, S represents the total entropy value, ∑ represents the sum, and p_i represents the energy on each frequency component divided by the total energy, thereby obtaining the energy probability on each frequency component;

[0168] The spectral energy value of each spectrum graph is obtained by the spectral entropy value calculation formula. After obtaining the spectral energy value of each spectrum graph, it is determined whether the spectral energy value of the spectrum graph of each speech frame is less than a preset threshold. The preset threshold can be determined according to the actual situation, that is, the spectral energy value of the spectrum graph of each speech frame is compared with the preset threshold.

[0169] When the spectrum energy value is less than the preset low threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset low threshold is greater than the preset high threshold, the first speech frame less than the preset low threshold is taken as the starting endpoint of the speech signal. In this embodiment, the preset low threshold and the preset high threshold can be determined according to the actual situation. That is to say, when the spectrum energy value is less than the preset low threshold, it means that the energy of the current speech frame in the frequency domain is very low, which may indicate a gap or a silent part of the speech signal; when the spectrum energy value of the adjacent speech frame is greater than the preset high threshold, immediately after the above low energy frame, the energy of one or more speech frames suddenly increases and exceeds the preset high threshold, which usually indicates the beginning of the speech signal. Detect the speech starting endpoint, traverse the speech frame sequence, and check the spectrum energy value of each frame. When it is found that the spectrum energy value of a frame is less than the preset low threshold, and the spectrum energy values ​​of the next or subsequent frames immediately following this frame are greater than the preset high threshold, the frame less than the low threshold is marked as the starting endpoint of the speech signal.

[0170] When the spectrum energy value is less than the preset high threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset high threshold is greater than the preset high threshold, the last speech frame greater than the preset high threshold is taken as the end point of the speech signal. That is to say, when the spectrum energy value is less than the preset high threshold, it means that the energy of the current speech frame in the frequency domain is relatively low, which may mean that the speech signal is weakening or about to end. When the spectrum energy value of the adjacent speech frame is greater than the preset high threshold, one or more frames have higher energy values ​​before the frame below the high threshold, exceeding the preset high threshold. When a series of low-energy frames appear after these high-energy frames, it is determined that the speech is really over. Detect the speech end point, traverse the speech frame sequence, and check the spectrum energy value of each frame from front to back. When a frame whose spectrum energy value is less than the preset high threshold is encountered, start recording the energy value of this frame and its subsequent frames. Continue to check the subsequent frames until a frame whose spectrum energy value is greater than the preset high threshold again is found, indicating that there may be more speech signals. If no frame with a value greater than the high threshold is found in subsequent frames, the last frame with a value less than the high threshold but immediately followed by one or more frames greater than the high threshold, i.e. the last frame in this series of low energy frames, is marked as the end endpoint of the speech signal.

[0171] By performing endpoint detection on speech signals, the accuracy of speech signal start and end endpoint detection can be significantly improved, false detection and missed detection can be reduced, which helps to reduce redundant data in subsequent speech processing and ensure the accuracy of speech signal recognition.

[0172] In one implementation of this embodiment, determining the user level of the target user according to the first target location and the itinerary information includes:

[0173] S710, determining the first time node of the target user according to the travel information;

[0174] S720, determining a path length from the first current location to the first target location according to the first target location and the first current location;

[0175] S730, obtaining the current time node;

[0176] S740, calculating the time period for the best matching robot to arrive at the first target location according to the path length and the preset speed of the best matching robot, and calculating the second time node for the best matching robot to arrive at the first target location according to the current time node and the time period;

[0177] S750, mapping the first path into a preset geographic data model, determining influencing factors of the first path, and determining a third time node for the optimal matching robot to arrive at the first target location according to the influencing factors;

[0178] S760, using a grey correlation analysis algorithm, constructing an evaluation matrix for the first time node, the second time node, and the third time node;

[0179] S770, determining the first time node, the second time node, and the weight value of the second time node;

[0180] S780, using a preset synthesis operator to synthesize the weight value and the evaluation matrix to obtain a comprehensive evaluation result of the user level;

[0181] S790. Determine the user level of the target user based on the comprehensive evaluation result of the user level.

[0182] According to the itinerary information, the first time node of the target user is determined. In this embodiment, the itinerary information may include the user's travel plan, departure time, arrival time, places passed, etc. That is to say, the first time node of the target user is determined through the itinerary information of the target user. In this embodiment, the first time node refers to the latest time for the target user to take the transportation tool. For example, if the target user's transportation tool is a train, the first time node is the latest time node for taking the train.

[0183] After determining the first time node of the target user, the path length from the first current location to the first target location is determined according to the first target location and the first current location. In other words, the path length for the target user to reach the first target location is determined by the first target location that the target user needs to reach and the current location of the target user. The path length for the target user to reach the first target location can be determined through indoor positioning technology, such as using wireless communication signals such as Wi-Fi, Bluetooth, etc. for indoor positioning, thereby determining the path length for the target user to reach the first target location.

[0184] The current time node is obtained, and the time period for the best matching robot to reach the first target location is calculated based on the path length and the preset speed of the best matching robot. That is, the time period for the best matching robot to reach the first target location is obtained by dividing the path length by the preset speed of the best matching robot through the previously obtained path length and the preset speed of the best matching robot. In this embodiment, the preset speed of the best matching robot can be determined according to the actual situation. The preset time period for the best matching robot to reach the first target location is obtained, and the second time node for the best matching robot to reach the first target location is calculated based on the current time node and the time period. For example, the current time node is 5:30 pm, and the preset time period for the best matching robot to reach the first target location is calculated to be 30 minutes. Therefore, the second time node is 6:00 pm.

[0185] The first path is mapped into a preset geographic data model, the influencing factors of the first path are determined, and the third time node for the optimal matching robot to arrive at the first target location is determined according to the influencing factors. In this embodiment, the influencing factors of the first path may be human traffic and roadblock factors. The preset geographic data model may be a Geodatabase data model. Geodatabase is a unified and intelligent spatial database based on a standard relational database management system. It is a core component in ArcGIS and is used to store and manage geographic spatial data.

[0186] After mapping the first path to the preset geographic data model and determining the influencing factors of the first path, the computer will automatically generate a time prediction formula. At this time, the time prediction formula will be displayed on the display. For example, the time prediction formula generated on the computer is:

[0187]

[0188] T represents time, L represents the path length, and V represents the robot's pre-set speed;

[0189] After obtaining the time prediction formula, the flow of people and roadblock factors are fitted into the time prediction formula, and the adjustment formula is obtained as follows:

[0190]

[0191] V represents the pre-set speed of the robot, L represents the path length, V1 represents the relationship between human flow and speed, and V2 represents the relationship between roadblocks and speed;

[0192] The relationship between traffic flow and speed is:

[0193]

[0194] Q represents the flow of people, and K represents the density of the flow of people;

[0195] The relationship between roadblocks and speed is:

[0196] V2=λ a x(l)

[0197] λ a represents the influence parameter of the roadblock on the robot speed, x(l) represents the number of roadblocks in the section with a path length of l;

[0198] Among them, λ a The influencing parameters are determined by factors such as the friction, temperature, and humidity of the road surface. In some cases, humidity can affect the friction coefficient between the robot and the ground or other objects. For example, a wet floor may cause the robot to slide or be difficult to maintain stability, which may affect the robot's speed. In addition, in a high temperature environment, the robot may need a more powerful cooling system to avoid overheating; in a low temperature environment, the robot may need a heating system to maintain its internal temperature. Therefore, in different road sections, λ a The values ​​of the influencing parameters are different. For example, the parameters in the road section are shown in Table 1:

[0199]

[0200] It should be noted that Table 1 is only a partial illustrative description and is not intended to limit the present application.

[0201] In a specific implementation, different influencing parameters such as the material friction, temperature and humidity of the ground can be input into the pre-trained model, and the λ corresponding to the material friction, temperature and humidity of the ground can be output. a Affects parameter values.

[0202] Finally, the adjusted time prediction formula is obtained:

[0203]

[0204] Among them, L represents the path length, V represents the pre-set speed of the robot, Q represents the flow of people, K represents the density of the flow of people, and λ a represents the influence parameter of the roadblock on the robot speed, x(l) represents the number of roadblocks in the section with a path length of l;

[0205] After obtaining the adjusted time prediction formula, the third time node of the robot is determined by the adjusted time prediction formula. In this embodiment, the third time node is the time node for the best matching robot to arrive at the first target location determined based on the influencing factors. For example, the second time node is 6:00 in the afternoon. The adjusted time prediction formula determines that the extended time period of the robot due to the influencing factors is 10 minutes. Therefore, the third time node is 6:10 in the afternoon.

[0206] The grey correlation analysis algorithm is used to construct the evaluation matrix of the first time node, the second time node and the third time node. Grey correlation analysis is a multivariate statistical analysis method based on grey system theory. It is mainly used to analyze the degree of correlation between the various factors in the system and can be used to construct an evaluation matrix. An evaluation matrix is ​​a tool used to evaluate and analyze the relationship between multiple variables or factors. It is usually presented in a structured manner to facilitate comparison and quantification of the importance, influence or performance of different factors. In other words, by constructing an evaluation matrix using the grey correlation analysis algorithm, the different importance of the first time node, the second time node and the third time node to the target user's time node can be determined, thereby determining the influencing factors of the time node. By analyzing the influencing factors, the urgency of the target user on the node can be obtained, thereby determining the user level of the target user.

[0207] The weight values ​​of the first time node, the second time node and the second time node can be determined by the hierarchical analysis method, which is a structured technology for complex decision-making problems and a decision analysis method that combines qualitative and quantitative methods. The eigenvalues ​​and eigenvectors of the judgment matrix are calculated by mathematical methods such as the eigenvector method, the arithmetic mean method, the geometric mean method, etc., and the eigenvectors are normalized to obtain the weight values ​​of each time node and related factors.

[0208] After the weight value is determined, the weight value is synthesized with the evaluation matrix using a preset synthesis operator to obtain the comprehensive evaluation result of the user level. In this embodiment, the preset synthesis operator can be a geometric mean type. The geometric mean type is the concept of the geometric mean, which refers to the nth root of the product of n observations. The geometric mean type is a way of calculating the average of a group of values. The weight values ​​of the first time node, the second time node, and the second time node are multiplied and synthesized with the evaluation matrix using the geometric mean type to obtain an evaluation result vector. Through the evaluation result vector, the membership of the evaluated object on each comment is further analyzed to obtain the comprehensive evaluation result of the user level. For example, the evaluation result vector is B=(b1, b2, b3, ... b m), the evaluation result vector B is normalized and the linear function is used for normalization, which is to linearly convert the original data into the range of [0,1] or [-1,1] to obtain the normalized data and determine the comprehensive evaluation result of the user level.

[0209] The user level of the target user is determined by the comprehensive evaluation result of the user level. That is to say, within the preset level evaluation threshold range, the comprehensive evaluation result of the user level is compared with the preset level evaluation threshold range to determine the level evaluation threshold range where the target user is located, thereby determining the user level of the target user.

[0210] By determining the user level of the target user, the user's urgency and needs can be assessed more accurately. Based on the user level, the service provider can allocate resources more effectively, which helps to improve service efficiency, ensure that resources are used reasonably, and improve user experience.

[0211] In one implementation of this embodiment, the robot information includes robot device information, and determining the best matching robot according to the user level, the first target location and the robot information includes:

[0212] S810, obtaining spatial location data of the robot;

[0213] S820, mapping the target user, the first target location, and the robot into a geographic data model, and displaying the model;

[0214] S830, determining a time threshold of the user in a preset user level database according to the user level, where the time threshold includes a first time period and a second time period, where the first time period is a time period between the first current location and the first target location, and the second time period is a time period between the second current location and the first current location;

[0215] S840, retrieving the target robot in the geographic data model according to the time threshold;

[0216] S850, determining environmental information of public places through a geographic data model;

[0217] S860, determining in real time the time and pre-consumed electric energy required for the target robot to reach the first target location via the location of the target user through environmental information of the public place;

[0218] S870, determining the power reserve of the target robot through the robot device information of the target robot;

[0219] S880: When the power reserve of the target robot is greater than the pre-consumed power and the time required for the target robot to reach the first target location via the first current location is less than a preset time threshold, determine the best matching robot.

[0220] The best matching robot is determined based on the user level, the first target location and the robot information. Specifically, first, the spatial location data of the robot is obtained. This can be done through lidar scanning or field measurement to obtain the spatial location data of the robot, such as the boundaries and range within the space, the shape of the building, the width of the road and other data.

[0221] Subsequently, the target user, the first target location, and the robot are mapped to a preset geographic data model and displayed. That is, the latitude and longitude coordinates of the target user are converted into the geographic data model, and the current location information of the robot is converted into the geographic data model and displayed in a visualization tool or platform, such as GIS software, map service, or customized map visualization application.

[0222] According to the user level, the user's time threshold is determined in a preset user level database, that is, through the user's level, the time threshold corresponding to the target user is determined in the preset user level database, wherein the time threshold includes a first time period and a second time period, the first time period being the time period required for the journey from the target user to the first target location, and the second time period being from the positioning of the best matching robot to the target user. By determining the two time periods and the user level, the user's time threshold is determined. For example, the target user's latest boarding time is 6:00, and there is still one hour left before the latest boarding time node. The time period required for the journey from the target user to the first target location is 30 minutes, and the time period from the positioning of the best matching robot to the target user is 20 minutes. At the same time, an extended time period is determined for the user in the preset user level database, and a time period is reserved for the user to encounter other influences on the road, thereby finally determining the user's time threshold.

[0223] The target robot is retrieved in the geographic data model through the time threshold. That is to say, based on the time when the best matching robot locates the target user and the user's time threshold, the target robot is obtained by determining in the geographic data model that the best matching robot locates the target user at a time less than the user's time threshold.

[0224] Through geographic data models, the environmental information of public places can be determined. GIS software can be used for spatial analysis, such as buffer analysis and overlay analysis, to determine the spatial distribution characteristics of indoor public places. Spatial analysis can also be used to evaluate environmental indicators such as accessibility and congestion of indoor public places.

[0225] After determining the environmental information of the public place, the time required for the target robot to reach the first target location via the location of the target user and the pre-consumed electric energy are determined in real time through the environmental information of the public place. Specifically, first, the precise coordinates of the location of the target user and the first target location need to be obtained, and the optimal path of the robot from the location of the target user to the first target location is planned using map data or an indoor navigation system. The time it takes for the target robot to reach the first target location is calculated based on the preset speed of the target robot. Subsequently, the battery capacity and energy consumption rate of the target robot and environmental information are obtained, such as the impact of environmental factors such as temperature and humidity on battery performance. The electric energy consumed by the target robot on the entire path is calculated based on the path length, the moving speed of the target robot, the energy consumption rate and environmental information. The energy consumption can be estimated using empirical formulas or simulation models.

[0226] After obtaining the time required for the target robot to reach the first target location via the target user's location and the pre-consumed electric energy, the target robot's electric energy reserve is determined through the target robot's robot device information. That is, the electric energy reserve is determined based on the target robot's robot device information. When the target robot's electric energy reserve is greater than the pre-consumed electric energy and the time required for the target robot to reach the first target location via the first current location is less than a preset time threshold, the optimal matching robot is determined. At this time, the target robot's electric energy reserve is sufficient to offset the pre-consumed electric energy, and the target robot also has sufficient time to lead the target user to the first target location.

[0227] By comprehensively considering the user level, target location and robot information, the most suitable robot can be accurately matched to the user, which can improve the efficiency of robot use, thereby ensuring the quality of service and improving user satisfaction and experience.

[0228] In one implementation of this embodiment, after determining the first target location that the target user needs to reach and performing face recognition on the face image to obtain the itinerary information of the target user, the method further includes:

[0229] S910, searching the geographic data model for a second target location closest to the target user;

[0230] S920, obtaining the destination of the target user according to the itinerary information of the target user, and determining whether the first target location and the second target location are the destination;

[0231] S930, if the first target location and the second target location are both destinations, generating second target location confirmation information to the target terminal according to the second target location;

[0232] S940, after receiving the confirmation information sent by the target terminal, taking the second target location as the first target location;

[0233] S950: After receiving the negative information sent by the target terminal, execute the steps of acquiring the robot information of each robot in real time and acquiring the first current location of the target user in real time.

[0234] Figure 2 A schematic diagram of a process for determining a target location provided in an embodiment of the present application;

[0235] The second target location closest to the target user is retrieved in the geographic data model. That is, the current location of the target user is determined, and the second target location closest to the target user is retrieved in the geographic data model. In this embodiment, the second target location and the first target location are both path locations that can reach the final destination. For example, there are two security checkpoints in the vehicle taken by the target user, and the second target location retrieved in the geographic data model is the security checkpoint close to the target user.

[0236] After the second target location is retrieved, the destination of the target user is obtained according to the itinerary information of the target user, and it is determined whether the first target location and the second target location are the destination. For example, the transportation vehicle taken by the target user and the security checkpoints where security checks can be performed are obtained according to the itinerary information of the target user. It is determined whether the first target location and the second target location are the destinations, that is, it is determined whether the first target location and the second target location are security checkpoints where security checks can be performed.

[0237] If the first target location and the second target location are both destinations, second target location confirmation information is generated and sent to the target terminal based on the second target location. When the first target location and the second target location are both security checkpoints that can be inspected, second target location confirmation information is generated and sent to the target terminal based on the second target location closest to the target user. The target user needs to confirm whether to replace the second target location with the first target location.

[0238] When the target user determines to replace the second target location with the first target location, after receiving the confirmation information sent by the target terminal, the second target location is used as the first target location, that is, the target user agrees to undergo security inspection at the nearest security checkpoint.

[0239] When the target user disagrees with replacing the second target location with the first target location, after receiving the negative information sent by the target terminal, the steps of obtaining the robot information of each robot in real time and the first current location of the target user in real time are executed. That is to say, when the target user disagrees with replacing the second target location with the first target location, for example, the target user needs to go to the first target location to handle other matters, the steps of obtaining the robot information of each robot in real time and the first current location of the target user in real time are executed.

[0240] Another implementation method in this embodiment is that for the airport check-in scenario, after obtaining the destination of the target user through the itinerary information of the target user, if it is found that the first target location (first security checkpoint) that the target user wants to reach cannot be inspected, but the second target location (second security checkpoint) can be inspected, a confirmation message is sent to the target user, and the content of the message can be "Second security checkpoint can be inspected, but the first security checkpoint cannot be inspected. Do you want to continue to the first security checkpoint?". When receiving the instruction from the target user to go to the second target location, the first target location that the target user wants to reach is corrected, and the first target location is replaced with the second target location. When receiving the instruction from the target user to go to the first target location, the step of obtaining the robot information of each robot in real time and obtaining the first current location of the target user in real time is continued. This application will not elaborate on this.

[0241] By introducing the retrieval and confirmation mechanism of the second target location, we can better understand user intentions and provide services that better meet user needs.

[0242] The embodiment of the present application further provides an interactive robot, each of which is connected to a central console, and each of which is pre-set with a first camera, and the interactive robot further includes:

[0243] a memory configured to store instructions; and

[0244] A processor is configured to retrieve instructions from memory and execute them.

[0245] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0246] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0247] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0248] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0249] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0250] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0251] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0252] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0253] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for controlling an intelligent interactive robot in a public place, characterized in that: Applied to a central console, the central console is connected to at least one robot, each of the robots is preset with a first camera, and the method includes: In response to receiving path navigation instructions from multiple user terminals, taking the user terminal corresponding to the path navigation instruction issued earliest as a target terminal; In response to receiving the voice information sent by the target terminal, sending a shooting instruction to the target terminal, wherein the shooting instruction is used to call the second camera of the target terminal to enable the target terminal to shoot a facial image of a target user corresponding to the target terminal; In response to receiving the face image sent by the target terminal, performing semantic recognition on the voice information to determine a first target location that the target user needs to reach, and performing face recognition on the face image to obtain itinerary information of the target user; acquiring robot information of each of the robots in real time, and acquiring a first current location of the target user in real time; determining a user level of the target user according to the first target location and the itinerary information; Determine the best matching robot according to the user level, the first target location, the information of each robot and the first current location, and obtain the second current location of the best matching robot; Based on the first target location, the first current location and the second current location, a path control plan for the optimal matching robot is generated and displayed.

2. The method according to claim 1, characterized in that The generating of the path control scheme of the optimal matching robot based on the first target location, the first current location and the second current location includes: generating a first path according to the second current location, the first current location, and the first target location, wherein the first path includes the second current location, the first current location, and the first target location, the second current location is a first waypoint of the first path, the first current location is a second waypoint of the first path, and the first target location is a third waypoint of the first path; A first control instruction is generated according to the first path, and the first control instruction is sent to the best matching robot so that the best matching robot executes the first control instruction.

3. The method according to claim 2, characterized in that The method further comprises: generating a first collection point according to the second current location and the first target location; Generate a second path through the first current location and the first collection point, the second path including the first current location and the first collection point, the first current location being a first waypoint of the second path, and the first collection point being a second waypoint of the second path; sending the second path to the target terminal; Generate a third path through the first set point and the second current location, the third path including the first set point and the second current location, the second current location being a first waypoint of the third path, and the first set point being a second waypoint of the third path; Generate a second control instruction according to the third path, and send the second control instruction to the best matching robot so that the best matching robot executes the second control instruction; Generate a fourth path through the first collection point and the first target location, the fourth path including the first collection point and the first target location, the first collection point being a first waypoint of the fourth path, and the first target location being a second waypoint of the fourth path; According to the fourth path, a third control instruction is generated, and the third control instruction is sent to the best matching robot, so that the best matching robot executes the third control instruction.

4. The method according to claim 3, characterized in that Before generating the third control instruction according to the fourth path, the method includes: determining the arrival time of the target user through the second path; Setting a preset waiting time threshold of the optimal matching robot according to the arrival time; Within a preset waiting time threshold, controlling the first camera of the optimal matching robot to capture an image; Calculating the similarity between the image and the face image, and if the similarity is greater than a preset similarity threshold, the optimal matching robot executes a third control instruction; If the waiting time of the best matching robot is greater than the preset waiting time threshold, and the similarity is less than the preset similarity threshold, the best matching robot sends a signal to exit the current service to the central console; In response to receiving the signal to exit the current service, a target parking area closest to the best matching robot is determined in a preset parking area, and the parking area position is sent to the best matching robot so that the best matching robot goes to the parking area position.

5. The method according to claim 1, characterized in that The performing semantic recognition on the voice information to determine the first target location that the target user needs to reach includes: Converting the voice information into a voice signal; Performing endpoint detection on the voice signal to determine the start endpoint and the end endpoint of the voice signal; Obtain an audio clip through the start endpoint and the end endpoint; Using a preset filtering algorithm on the audio segment to obtain a filtered audio segment; Inputting the filtered audio segment into a pre-trained acoustic model to obtain a feature vector of the audio segment; Inputting the filtered audio segment into a pre-trained language model to obtain a phrase sequence of the audio segment; Performing semantic understanding on the audio clip through the feature vector and the phrase sequence to obtain key information of the audio clip; The first target location that the target user needs to reach is determined through the key information.

6. The method according to claim 5, characterized in that The performing endpoint detection on the voice signal to determine the start endpoint and the end endpoint of the voice signal includes: Performing frame processing on the speech signal to obtain a plurality of speech frames of the speech signal; Performing Fourier transform on each of the speech frames to obtain a spectrogram of each of the speech frames; Calculating the spectrum energy value of each of the spectrum graphs; When the spectrum energy value is less than a preset lower threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset lower threshold is greater than a preset upper threshold, the first speech frame whose spectrum energy value is less than the preset lower threshold is taken as the starting endpoint of the speech signal; When the spectrum energy value is less than a preset high threshold and the spectrum energy value of the speech frame adjacent to the speech frame whose spectrum energy value is less than the preset high threshold is greater than the preset high threshold, the last speech frame whose spectrum energy value is greater than the preset high threshold is taken as the end endpoint of the speech signal.

7. The method according to claim 2, characterized in that The determining the user level of the target user according to the first target location and the itinerary information includes: Determine the first time node of the target user according to the travel information; Determining a path length from the first current location to the first target location according to the first target location and the first current location; Get the current time node; Calculate the time period for the best matching robot to reach the first target location according to the path length and the preset speed of the best matching robot, and calculate the second time node for the best matching robot to reach the first target location according to the current time node and the time period; Mapping the first path into a preset geographic data model, determining influencing factors of the first path, and determining a third time node for the optimal matching robot to arrive at the first target location according to the influencing factors; Using a grey correlation analysis algorithm, constructing an evaluation matrix for the first time node, the second time node, and the third time node; Determine the first time node, the second time node, and a weight value of the second time node; The weight value and the evaluation matrix are synthesized by using a preset synthesis operator to obtain a comprehensive evaluation result of the user level; The user level of the target user is determined based on the comprehensive evaluation result of the user level.

8. The method according to claim 7, characterized in that The robot information includes robot device information, and determining the best matching robot according to the user level, the first target location and the robot information includes: Acquire spatial location data of the robot; Mapping the target user, the first target location, and the robot into the geographic data model and displaying them; According to the user level, determining the time threshold of the user in a preset user level database, the time threshold comprising a first time period and a second time period, the first time period being a time period between a first current location and a first target location, and the second time period being a time period between a second current location and the first current location; Retrieving a target robot in the geographic data model according to the time threshold; Determining environmental information of the public place through the geographic data model; Determine in real time the time and power consumption required for the target robot to reach the first target location via the location of the target user through the environmental information of the public place; Determining the power reserve of the target robot through the robot device information of the target robot; When the power reserve of the target robot is greater than the pre-consumed power and the time required for the target robot to reach the first target location via the first current location is less than a preset time threshold, the optimal matching robot is determined.

9. The method according to claim 8, characterized in that After determining the first target location that the target user needs to reach and performing face recognition on the face image to obtain the itinerary information of the target user, the method further includes: Retrieving a second target location closest to the target user in the geographic data model; Obtaining the destination of the target user according to the itinerary information of the target user, and determining whether the first target location and the second target location are the destination; If the first target location and the second target location are both the destination, generating second target location confirmation information to the target terminal according to the second target location; After receiving the confirmation information sent by the target terminal, taking the second target location as the first target location; After receiving the negative information sent by the target terminal, the steps of acquiring the robot information of each robot in real time and acquiring the first current location of the target user in real time are performed.

10. An interactive robot, applied to the robot according to any one of claims 1 to 9, characterized in that: Each of the interactive robots is connected to the central console, and each of the interactive robots is preset with a first camera, and the interactive robots also include: a memory configured to store instructions; and The processor is configured to call the instructions from the memory and execute them.

Images