Notification method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to the technical field of robots, and discloses a notification method, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining environment information collected by a robot in a delivery process, determining an abnormal scene type according to the environment information, adjusting preset initial notification time according to the abnormal scene type to obtain target notification time, and controlling the robot to go to a target position to execute notification operation according to the target notification time. According to the robot provided by the embodiment of the invention, the abnormal scene type can be reliably identified, and the initial notification time is adjusted according to the abnormal scene type so as to execute the notification operation, so that the arrival time of the robot and the food taking time of a customer can be coordinated and synchronized, and waiting of the customer is avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robotics technology, and in particular, to a notification method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of robot technology, robots are widely used in various service places to provide customers with services such as guiding and delivering items. Robots often encounter abnormal situations during the delivery process, resulting in the robot failing to deliver items to customers at the scheduled time, but notifying customers to pick up the goods at the scheduled time, causing customers to go out too early to wait for the items or go out too late to pick up the items, thereby reducing the user experience. Summary of the invention

[0003] One purpose of the embodiments of the present application is to provide a notification method, an electronic device, and a computer-readable storage medium to solve the technical problem that the related art cannot flexibly adjust the notification time under abnormal circumstances.

[0004] In a first aspect, an embodiment of the present application provides a notification method, applied to a robot, comprising:

[0005] Obtain environmental information collected by the robot during the delivery process;

[0006] Determine the abnormal scene type according to the environmental information;

[0007] Adjusting the preset initial notification time according to the abnormal scenario type to obtain a target notification time;

[0008] The robot is controlled to go to a target location to perform a notification operation according to the target notification time.

[0009] Optionally, controlling the robot to go to a target location to perform a notification operation according to the target notification time includes:

[0010] Detecting whether the robot has reached a preset target position and whether the target notification time has arrived, and obtaining a spatiotemporal detection result;

[0011] In response to the spatiotemporal detection result, the robot is controlled to perform a notification operation.

[0012] Optionally, the robot is configured with at least two types of wireless communication modules;

[0013] The step of controlling the robot to perform a notification operation in response to the spatiotemporal detection result includes:

[0014] In response to the spatiotemporal detection result, detecting whether the target wireless communication module can communicate with the cloud, and obtaining a communication detection result, wherein the target wireless communication module is one of the at least two types of wireless communication modules;

[0015] The robot is controlled to perform a notification operation according to the communication detection result.

[0016] Optionally, the notification operation includes a cloud notification operation and / or a local broadcast operation;

[0017] The controlling the robot to perform a notification operation according to the communication detection result comprises:

[0018] In response to the communication detection result, controlling the robot to perform a cloud notification operation; and / or,

[0019] In response to the communication detection result, the robot is controlled to perform a local broadcast operation.

[0020] Optionally, the communication detection result includes a normal communication result or an abnormal communication result, and the cloud notification operation includes a cloud direct connection notification operation or a cloud forwarding notification operation;

[0021] In response to the communication detection result, controlling the robot to perform a cloud notification operation includes:

[0022] If the communication detection result is a normal communication result, controlling the target wireless communication module to execute a cloud direct connection notification operation for sending a user notification command to the cloud, so that the cloud sends an item arrival notification to the target device according to the user notification command;

[0023] If the communication detection result is a communication abnormality result, the candidate wireless communication module is controlled to execute a cloud forwarding notification operation on sending a user notification command to an available device, so that the available device triggers the cloud to send an item arrival notification to the target device according to the user notification command. The candidate wireless communication module is another wireless communication module of at least two types of wireless communication modules, and the available device is a device that can establish a communication connection with the robot and communicate with the cloud.

[0024] Optionally, in response to the communication detection result, controlling the robot to perform a local broadcast operation includes:

[0025] In response to the communication detection result, obtaining environmental noise information and loudness-related information collected by the robot when the robot is at the target position, wherein the loudness-related information is used to constrain the loudness of the robot when performing a local broadcast operation;

[0026] Determine a target broadcast loudness according to the environmental noise information and the loudness association information;

[0027] The robot is controlled to perform a local broadcast operation according to the target broadcast loudness.

[0028] Optionally, in response to the spatiotemporal detection result, controlling the robot to perform a notification operation includes:

[0029] If the spatiotemporal detection result is used to indicate that the robot has reached a preset target position and the target notification time has arrived, controlling the robot to perform a notification operation at the target notification time;

[0030] If the spatiotemporal detection result is used to indicate that the robot has reached a preset target position, and the target notification time has not arrived, then obtaining the arrival time of the robot at the target position, and controlling the robot to perform a notification operation at the arrival time;

[0031] If the spatiotemporal detection result indicates that the robot has not reached the preset target position, and the target notification time has arrived, then controlling the robot to perform a notification operation after reaching the target position;

[0032] If the spatiotemporal detection result is used to indicate that the robot has not reached the preset target position and the target notification time has not arrived, then return to execute the step of detecting whether the robot has reached the preset target position and whether the target notification time has arrived to obtain the spatiotemporal detection result and subsequent steps.

[0033] Optionally, adjusting a preset initial notification time according to the abnormal scenario type to obtain a target notification time includes:

[0034] Determine the time deviation corresponding to the abnormal scene type;

[0035] A target notification time is determined according to the time deviation and the initial notification time.

[0036] In a second aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the electronic device implements the above-mentioned notification method.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned notification method.

[0038] The embodiments of the present application can achieve the following technical effects: in the notification method provided in the embodiments of the present application, the environmental information collected by the robot during the delivery process is obtained, the abnormal scene type is determined according to the environmental information, the preset initial notification time is adjusted according to the abnormal scene type, the target notification time is obtained, and the robot is controlled to go to the target location to perform the notification operation according to the target notification time. The robot provided in the embodiments of the present application can reliably identify the abnormal scene type and adjust the initial notification time according to the abnormal scene type in order to perform the notification operation, thereby ensuring that the arrival time of the robot and the customer's meal pick-up time can be coordinated and synchronized with each other to avoid customers waiting. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A schematic diagram of the system architecture of a notification system provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a robot provided in an embodiment of the present application;

[0042] Figure 3 A circuit principle block diagram of a robot provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the notification system provided by the embodiment of the present application in the hotel meal delivery scenario provided by the embodiment of the present application;

[0044] Figure 5 A flowchart of a notification method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of a scene in which multiple people are waiting for an elevator provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of a scenario for generating environment description information based on a preset big data model provided in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of the structure of a notification device provided in an embodiment of the present application;

[0048] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0050] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0051] This application embodiment provides a notification system, please refer to Figure 1 The notification system 100 includes a server 200 , a relay device 300 and a robot 400 , and the server 200 is communicatively connected with the robot 400 via the relay device 300 .

[0052] The server 200 serves as the control core of the notification system 100 and is used to send tasks to the robot 400 or schedule the robot 400 or receive data uploaded by the robot 400.

[0053] The relay device 300 is connected to the server 200 in communication, and the communication connection includes a wired connection or a wireless connection, the wired connection includes an Ethernet connection or an optical fiber connection, and the wireless connection includes a WI FI connection or a 6G connection or a 5G connection or a 4G connection, etc. The relay device 300 serves as an information medium between the server 200 and the robot 400, and is used to send various data or commands transmitted by the server 200 to the robot 400, or to send various data or commands transmitted by the robot 400 to the server 200.

[0054] The robot 400 is used to perform tasks, including meal delivery tasks, shopping guide tasks, etc. Among them, the robot 400 can be a delivery robot, a shopping guide robot, a guiding robot or a cleaning robot, etc.

[0055] Please also read Figure 2 and Figure 3 The robot 400 includes a shell 41, a camera module 42, a laser radar 43, a display screen 44, a charging circuit 45, a walking component 46, a wireless communication module 47, an inertial measurement unit 48 and a microcontroller 49.

[0056] The housing 41 is configured to be capable of carrying items. Figure 2 As shown, the housing 41 includes a chassis structure 411 and a bearing structure 412, wherein the bearing structure 412 is mounted on the chassis structure 411. The bearing structure 412 includes at least two brackets 4121 and at least one tray 4122, wherein one end of the bracket 4121 is mounted on the chassis structure 411, and the other end of the bracket 4121 is arranged away from the chassis structure 411, and the tray 4122 is arranged between at least two brackets 4121. Food or other items can be placed on the tray 4122.

[0057] The camera module 42 is disposed on the housing 41 and is used to collect environmental images of the surrounding environment.

[0058] The laser radar 43 is disposed on the housing 41 and is used to collect radar data of the surrounding environment.

[0059] The display screen 44 is disposed on the housing 41 and is used for interacting with the user and presenting an interactive interface to the user. The interactive interface includes various UI buttons that can interact with the user.

[0060] The charging circuit 45 is provided on the housing 41 and is used to receive the power provided by the charging pile 400. The charging circuit 45 includes a charging electrode and a charging processing circuit. The charging electrode is used to dock with the supply electrode of the charging pile 400. The charging processing circuit is used to store the power provided by the charging pile 400 and provide it to various devices for operation. When the supply electrode of the charging pile 400 docks with the charging electrode, the power provided by the charging pile 400 can be provided to the robot 400 through the supply electrode and the charging electrode.

[0061] The walking assembly 46 is mounted on the chassis structure 411 of the housing 41, and is used to drive the chassis structure 411 to drive the bearing structure 412 to move. The walking assembly 46 includes a walking wheel and a motor, and the motor is used to drive the walking wheel to rotate.

[0062] The wireless communication module 47 is installed in the housing 41 and is used for wireless communication with the server 200 .

[0063] The inertial measurement unit 48 is installed in the housing 41 and is used to detect the posture of the robot 400 .

[0064] The microcontroller 49 is respectively connected to the camera module 42, the lidar 43, the display screen 44, the charging circuit 45, the walking component 46, the wireless communication module 47 and the inertial measurement unit 48, wherein the microcontroller 47 serves as the control core of the robot 400 and is used to control the robot 400 to process and analyze various business logics.

[0065] The robot 300 can detect whether the robot 300 is lifted off the ground with malicious intent.

[0066] In some embodiments, when the microcontroller 38 calls the environmental image captured by the camera module 32, the environmental image is input into the pre-trained machine posture model to obtain machine posture information. If the machine posture information is used to indicate that the robot is maliciously lifted off the ground, the microcontroller 38 generates abnormal lift-off information.

[0067] In some embodiments, the robot 300 further includes an infrared transceiver, which is disposed at the bottom of the chassis structure 311, and is used to transmit infrared signals to the ground and receive infrared signals reflected back by the ground. The microcontroller 38 is electrically connected to the infrared transceiver, and is used to receive the infrared signal sent by the infrared transceiver, and determine the relative height of the robot from the ground according to the intensity of the infrared signal. When the user picks up the robot, the relative height of the robot from the ground becomes higher and higher, and the intensity of the infrared signal decreases as the relative height increases. When the user puts the robot on the ground, the relative height of the robot from the ground becomes lower and lower, and the intensity of the infrared signal increases as the relative height decreases. Therefore, when the microprocessor 38 detects that the intensity of the infrared signal first decreases and then increases, it can be determined that the robot is maliciously lifted off the ground.

[0068] In order to explain in detail the working principle of the notification system provided by the embodiment of the present application, the embodiment of the present application combines Figure 4 The detailed description is as follows:

[0069] See also Figure 4 The notification system provided in the embodiment of the present application is applied in the hotel meal delivery scenario. A preset waiting area 41 is set in the hotel lobby, and multiple robots 400 are waiting in the preset waiting area 41. Multiple relay devices 300 are respectively set on each floor of the hotel. Figure 4 , a gray circle is used to represent the robot 400 .

[0070] When the server 200 receives a task trigger command, the server 200 responds to the task trigger command, selects a specified task from the task set 43, and detects whether there is a robot 400 suitable for performing the specified task in the preset waiting area 41. If there is, the server 200 sends a task execution command to the robot 400, and the robot 400 responds to the task execution command and performs the specified task. If there is no robot 400, the server 200 dispatches a suitable robot from the remaining robots to perform the specified task. The specified task is: the server 200 notifies the robot 400 to deliver breakfast to room 206 on the second floor.

[0071] The server 200 calculates the meal delivery time of the robot 400 from the preset waiting area 41 to the room 206 on the second floor according to the designated task, sets the meal delivery time as the initial notification time, and sends the initial notification time to the robot 400, so that when the initial notification time arrives, the robot 400 notifies the guest in the room 206 on the second floor to go out and pick up the meal.

[0072] The robot 400 takes the elevator 42 with breakfast to the room 206 on the second floor. After leaving the elevator 42 , the robot 400 needs to walk along a long corridor 43 to reach the room 206 on the second floor.

[0073] In the process of going to Room 206 on the second floor, the robot 400 is likely to encounter various abnormal events. For example, in the process of going to the elevator 43, there are many people queuing to get on the elevator, and the robot 400 needs to wait in line to get on the elevator. For another example, the robot 400 is likely to encounter a cleaning car or other objects blocking the way to Room 206 on the second floor in the corridor 43. Cleaning staff usually need to push the cleaning car to clean the checked-out rooms. For another example, the robot 400 encounters the occupants of the next room blocking the way in the corridor 43. For another example, the robot 400 is maliciously moved by others, which quickly shortens the time to reach Room 206 on the second floor.

[0074] It is understandable that when the robot delivers food in a hotel, it is easy to be blocked and difficult to bypass in the elevator and guest room corridor, because the robot can only go to the corresponding floor by elevator, and cannot bypass to the corresponding floor through other places. The guest room corridor is usually a closed section at one end, and cannot bypass to the guest room corridor through other corridors or other locations. Therefore, when the robot encounters obstacles in the above two places, it is not easy to bypass, and usually waits for the obstacles to be removed before continuing the delivery task. When the robot encounters obstacles and is blocked in other places, it can plan an avoidance path to avoid the obstacle, and it does not excessively affect the normal progress of the delivery task.

[0075] When the robot provided by the related technology encounters the above-mentioned abnormal events, it still notifies the user to pick up the food according to the fixed delivery time. This causes the guests to stand at the door in advance waiting for the robot to deliver the food, causing the taste of the food to deteriorate due to the extended time. Alternatively, the robot arrives at the door in advance but notifies the guests to pick up the food later, causing the guests to fail to enjoy the joy of the early arrival of the food.

[0076] When the robot provided in the embodiment of the present application encounters the above-mentioned abnormal events, it can reliably identify the abnormal scene type and adjust the initial notification time according to the abnormal scene type, thereby ensuring that the arrival time of the robot and the customer's meal pick-up time can be coordinated and synchronized with each other to avoid guests waiting.

[0077] The following is a notification method provided by the present application embodiment, which is applied to a robot. Figure 5The notification method includes the following steps S51-S54:

[0078] S51: Obtaining environmental information collected by the robot during the delivery process.

[0079] In this step, environmental information is used to represent the environmental conditions of the robot during the delivery process. The environmental information includes environmental images taken by the camera, posture data collected by the inertial detection unit, radar data, etc.

[0080] S52: Determine the abnormal scene type according to the environmental information.

[0081] In this step, the abnormal scenario type is used to indicate the type of abnormal scenario encountered by the robot during the delivery process. Please refer to Table 1:

[0082] Table 1

[0083]

[0084] As shown in Table 1, abnormal scenarios include the robot being blocked by others in front of the elevator, the robot being blocked by a cleaning vehicle in the corridor, the robot being blocked by others in the corridor, the robot being hijacked and moving forward, the robot being hijacked and moving backward, etc. Correspondingly, the abnormal scenario types are the first abnormal scenario type in which the robot is waiting in front of the elevator, the second abnormal scenario type in which the robot is blocked by a cleaning vehicle in the corridor, the third abnormal scenario type in which the robot is blocked by a rebel in the corridor, the fourth abnormal scenario type in which the robot is hijacked and moving forward, the fifth abnormal scenario type in which the robot is hijacked and moving backward, etc. It is understandable that the designer can match other abnormal scenario types for the robot based on engineering experience, and is not limited to the above-mentioned various abnormal scenario types.

[0085] Determining the type of abnormal scene based on environmental information includes the following steps: determining whether the robot is in a blocked state based on the environmental information; if the robot is in a blocked state, acquiring an environmental image; determining the type of abnormal scene based on the environmental image; if the robot is not in a blocked state, detecting whether the robot is in a hijacked state; if the robot is in a hijacked state, determining the current position of the robot; determining the type of abnormal scene based on the current position and target position of the robot; if the robot is not in a hijacked state, maintaining the robot's motion state.

[0086] The environmental information includes radar data. Determining whether the robot is in a blocked state based on the environmental information includes the following steps: determining the position information of each obstacle and the relative distance between each obstacle and the robot based on the radar data; detecting whether there is an obstacle on the preset delivery path of the robot based on the position information of each obstacle; if so, determining that the robot is in a blocked state; if not, determining that the robot is not in a blocked state.

[0087] Determining the abnormal scene type according to the environmental image includes the following steps: determining environmental description information about the environmental image according to the environmental image, and determining the abnormal scene type according to the environmental description information.

[0088] Determining environmental description information about an environmental image based on an environmental image includes the following steps: generating a first prompt word based on the environmental image, inputting the first prompt word into a preset big data model so that the preset big data model outputs environmental description information, wherein the first prompt word is used to prompt the preset big data model to make a scene description of the environmental image.

[0089] Generating a first prompt word according to an environmental image includes the following steps: constructing a prompt instruction, and encapsulating the prompt instruction and the image into a first prompt word. Among them, the first prompt word is a prompt term, and prompt is a prompt word of an artificial intelligence model, which is a method of using natural language to guide or inspire an artificial intelligence model to complete a specific task. The prompt term can prompt the artificial intelligence model with the context of the input information and the parameter information of the input model. When training a supervised learning or unsupervised learning model, the prompt term can help the artificial intelligence model better understand the intention of the input and respond accordingly.

[0090] The first prompt word is used to indicate the user's question about the preset big data model processing environment image. In the first prompt word, the first prompt word declares what the preset big data model needs to do, the format of the output content, and the environment description information provided for the preset big data model.

[0091] The expression of the first prompt word is: ins = "Suppose you are a food delivery robot, please give the environment description information of the following image, the output format is: 'scene description': 'description summary'. As shown in the following formula: F = LLM (i ns + Z), F is the environment description information, LLM is the preset big data model, i ns is the prompt instruction, and Z is the environment image. The preset big data model can be an existing commercial big model, such as the GPT-4 big model.

[0092] Determining the abnormal scene type according to the environmental description information includes the following steps: extracting target keywords from the environmental description information, determining the environmental scene of the robot according to the target keywords, and determining the abnormal scene type according to the environmental scene.

[0093] The robot is equipped with a keyword library, which includes multiple benchmark keywords set in advance, such as elevator, floor, waiting, playing with mobile phone, carrying bag, riding, entering and exiting, corridor, cleaning car, cleaning staff, cleaning aunt, cleaning uncle, etc. The embodiment of the present application extracts each phrase from the environment description, searches the phrase in the keyword library, and determines whether there is a benchmark keyword corresponding to the phrase in the keyword library. If so, the phrase is used as the target keyword. If not, the next phrase is traversed and compared with the keyword library.

[0094] The environmental scenes include elevator scenes and corridor scenes. Determining the environmental scene in which the robot is located based on the target keywords includes the following steps: determining a first number of target keywords used to describe the elevator scene and a second number of target keywords used to describe the corridor scene. If the first number is greater than or equal to the second number, then determining that the environmental scene in which the robot is located is the elevator scene; if the first number is less than the second number, then determining that the environmental scene in which the robot is located is the corridor scene.

[0095] In some embodiments, determining the abnormal scene type according to the environmental scene includes the following steps: if the environmental scene is an elevator scene, generating a second prompt word, and inputting the second prompt word into a preset big data model so that the preset big data model outputs the total number of people waiting for the elevator, wherein the second prompt word is used to prompt the preset big data model to output the total number of people waiting for the elevator based on the environmental image; if the total number of people is greater than the preset number of people, the abnormal scene type is determined to be the first abnormal scene type; if it is less than or equal to the preset number of people, it is determined that the robot is in a normal waiting state.

[0096] The expression of the second prompt word is: ins = "Please give the total number of people waiting for the elevator in the above image. The output format is: 'total number of people'.

[0097] In some embodiments, determining the type of abnormal scene based on the environmental scene includes the following steps: if the environmental scene is a corridor scene, generating a third prompt word, inputting the third prompt word into a preset big data model, so that the preset big data model outputs subject information that blocks the robot, the subject information is used to represent the subject that blocks the robot in the corridor area, and determining the type of abnormal scene based on the subject information.

[0098] The expression of the third prompt word is: ins = "Please give the object that blocks the robot's progress in the above image. The output format is: 'object'.

[0099] The subject information includes cleaning vehicle information and bystander information. Determining the abnormal scene type based on the subject information includes the following steps: if the subject information is cleaning vehicle information, determining the abnormal scene type is the second abnormal scene type; if the subject information is bystander information, determining the abnormal scene type is the third abnormal scene type.

[0100] Detecting whether the robot is in a hijacked state includes the following steps: obtaining gyroscope data, the gyroscope data includes Z-axis acceleration data, and determining whether the change rate of the Z-axis acceleration data is greater than a preset change threshold. If so, it is determined that the robot is in a hijacked state; if so, it is determined that the robot is in a normal operating state.

[0101] Determining the abnormal scene type according to the current position and target position of the robot includes the following steps: determining the first position of the robot before being hijacked, respectively calculating the first distance from the first position of the robot to the target position, and the second distance from the current position of the robot to the target position, if the first distance is greater than the second distance, determining the abnormal scene type as the fourth abnormal scene type, if the first distance is less than the second distance, determining the abnormal scene type as the fifth abnormal scene type. The target position is the end point of the robot's meal delivery, as mentioned above, the target position is the door of Room 206 on the second floor.

[0102] Please also read Figure 6 and Figure 7 For example, when the robot 400 is waiting for the elevator 42 in front of the elevator waiting area, there are many people who need to get on the elevator standing in front of the robot 400. At this time, the robot 400 detects that it encounters an obstacle (that is, it encounters a person who is very close to the robot), and the robot 400 is in a blocked state. Then, the robot 400 takes a picture of the people in front of it, obtains an environmental image 61, generates a first prompt word 62 based on the environmental image, encapsulates the first prompt word 62 into a data analysis request 63, and sends the data analysis request 63 to the server 200. The server 200 parses the first prompt word 62 from the data analysis request 63, inputs the first prompt word 62 into a preset big data model 64, and the preset big data model 64 responds to the first prompt word 62, performs scene analysis on the environmental image, and finally obtains the following environmental description information 65:

[0103] Scene description: As a food delivery robot, I can give the following description of the environment scene based on the picture information provided.

[0104] “The environment I am in is a busy and orderly public area, next to a modern gray building. Two closed silver elevator doors become the visual focus. The characters waiting for the elevators each occupy a different position, and their clothes and accessories are different, reflecting different personalities and identities. From casual T-shirts to formal suits, from lightweight backpacks to practical handbags, these details enrich the layering of the picture, allowing the viewer to feel the diversity and authenticity of the scene.

[0105] The whole scene is filled with a sense of calmness in waiting. Although there may be a little anxiety and expectation in the air, everyone maintains due courtesy and order, creating a harmonious and orderly environment. As a food delivery robot, I will also follow this atmosphere and deliver warmth and convenience to every customer with the fastest speed and the best service."

[0106] Description summary: "This picture successfully creates a scene of waiting for the elevator that is full of life and modernity through delicate depiction and clever composition."

[0107] The embodiment of the present application searches the description summary in the environment description information and respectively searches out the following target keywords "elevator" and "waiting". Therefore, the embodiment of the present application determines that the environment scene is an elevator scene.

[0108] Next, the embodiment of the present application continues to generate a second prompt word: In the above picture piece, how many people are waiting for the elevator.

[0109] The total number of people waiting for the elevator is output by the preset big data model: In the above picture, the total number of people waiting for the elevator is 4. They are standing in front of the gray building, facing two closed silver elevator doors, waiting for the arrival of the elevator with different expressions.

[0110] If the preset number threshold is 8, since the total number of people is less than the preset number threshold, the embodiment of the present application determines that the abnormal scene type is the first abnormal scene type.

[0111] This embodiment utilizes the characteristics of the preset big data model that is good at analyzing and describing images. By constructing various prompt words and transmitting each prompt word to the preset big data model, the preset big data model can output reliable and rich environmental description information so that the type of abnormal scene can be accurately, reliably and quickly determined.

[0112] S53: Adjust the preset initial notification time according to the abnormal scenario type to obtain the target notification time.

[0113] In this step, the embodiment of the present application determines the time deviation corresponding to the abnormal scenario type, and determines the target notification time according to the time deviation and the initial notification time.

[0114] In some embodiments, determining the time deviation corresponding to the abnormal scenario type includes the following steps: if the abnormal scenario type is the first abnormal scenario type, dividing the total number of people by the preset number of people to obtain a result, and if the result has a remainder, adjusting the result according to the further method to obtain the shifts in which the robot can get on the elevator, and multiplying the shifts by the preset duration of each shift to obtain the time deviation. In the embodiment of the present application, the time deviation is added to the initial notification time to obtain the target notification time.

[0115] For example, if the total number of people is 4, the preset capacity is 8, and the preset shift duration is 5 minutes, then the shift is 1 and the time deviation is 5 minutes. If the total number of people is 12, then the shift is 2 and the time deviation is 10 minutes.

[0116] In some embodiments, determining the time deviation corresponding to the abnormal scenario type includes the following steps: if the abnormal scenario type is the second abnormal scenario type, obtaining a first preset duration corresponding to the second abnormal scenario type, and using the first preset duration as the time deviation. In the embodiment of the present application, the time deviation is added to the initial notification time to obtain the target notification time.

[0117] For example, it usually takes the cleaning staff 20 minutes to clean the guest rooms. Then 20 minutes is the first preset time, and the time deviation is 20 minutes. At this time, the robot will broadcast a voice message to urge the cleaning staff to move the cleaning cart so that the robot can pass.

[0118] In some embodiments, determining the time deviation corresponding to the abnormal scenario type includes the following steps: if the abnormal scenario type is a third abnormal scenario type, obtaining a second preset duration corresponding to the third abnormal scenario type, and using the second preset duration as the time deviation. In this embodiment of the application, the time deviation is added to the initial notification time to obtain the target notification time.

[0119] For example, a guest in the next room to Room 206 on the second floor blocks the robot in the corridor. The guest in this situation affects the robot's passage time by 3 minutes. Therefore, 3 minutes is the second preset time length, and the time deviation is 3 minutes. At this time, the robot will broadcast a voice to urge the guest to give way so that the robot can pass.

[0120] In some embodiments, determining the time deviation corresponding to the abnormal scene type includes the following steps: if the abnormal scene type is the fourth abnormal scene type, obtaining the absolute value of the difference between the first distance and the second distance, dividing the absolute value of the difference by the speed of the robot to obtain the time deviation. In the embodiment of the present application, the initial notification time is subtracted from the time deviation to obtain the target notification time.

[0121] In some embodiments, determining the time deviation corresponding to the abnormal scene type includes the following steps: if the abnormal scene type is the fifth abnormal scene type, obtaining the absolute value of the difference between the first distance and the second distance, dividing the absolute value of the difference by the speed of the robot to obtain the time deviation. In the embodiment of the present application, the initial notification time is added to the time deviation to obtain the target notification time.

[0122] S54: Control the robot to go to the target location to perform the notification operation according to the target notification time.

[0123] The robot provided in the embodiment of the present application can reliably identify the abnormal scene type and adjust the initial notification time according to the abnormal scene type to perform the notification operation, thereby ensuring that the robot's arrival time and the customer's meal pick-up time can be coordinated and synchronized with each other to avoid guests waiting.

[0124] In some embodiments, controlling the robot to go to the target location to perform the notification operation according to the target notification time includes the following steps S541-S542:

[0125] S541: Detect whether the robot has reached the preset target position and whether the target notification time has arrived, and obtain the spatiotemporal detection result.

[0126] S542: In response to the spatiotemporal detection result, control the robot to perform a notification operation.

[0127] In S541, the target location is the destination of the robot delivery, for example, the target location is the destination of the robot delivering food. The spatiotemporal detection result is the robot's location detection result and the delivery time detection result. The location detection result is used to indicate whether the robot has reached the target location, and the delivery time detection result is used to indicate whether the target notification time has arrived.

[0128] In S542, in response to the spatiotemporal detection result, controlling the robot to perform a notification operation includes the following steps: if the spatiotemporal detection result is used to indicate that the robot has reached a preset target position and the target notification time has arrived, then controlling the robot to perform a notification operation at the target notification time; if the spatiotemporal detection result is used to indicate that the robot has reached a preset target position and the target notification time has not arrived, then obtaining the arrival time of the robot at the target position, and controlling the robot to perform a notification operation at the arrival time; if the spatiotemporal detection result is used to indicate that the robot has not reached the preset target position and the target notification time has arrived, then controlling the robot to perform a delivery abnormality notification operation according to a preset time alarm threshold; if the spatiotemporal detection result is used to indicate that the robot has not reached the preset target position and the target notification time has not arrived, then returning to the step of detecting whether the robot has reached the preset target position and whether the target notification time has arrived to obtain the spatiotemporal detection result and subsequent steps. In this way, the embodiment of the present application provides a time monitoring basis for the robot to perform notification operations and delivery abnormalities by adjusting the initial notification time in advance to obtain the target notification time, thereby ensuring that the robot can reliably deliver items to users and avoid false triggering of notification operations.

[0129] Controlling the robot to perform a delivery abnormality notification operation according to a preset time alarm threshold includes the following steps: when the target time has arrived and the robot has not reached the preset target position, performing a delay operation to determine whether the current time is greater than the preset time alarm threshold; if so, the robot is controlled to perform a delivery abnormality notification operation; if so, the robot is controlled to perform a notification operation.

[0130] The robot is configured with at least two types of wireless communication modules. During the process of executing the notification operation, the embodiment of the present application detects whether the target wireless communication module can communicate with the cloud to obtain a communication detection result, wherein the target wireless communication module is one of the at least two types of wireless communication modules, and the robot is controlled to execute the notification operation according to the communication detection result.

[0131] The communication detection results include normal communication results and abnormal communication results. The normal communication results are used to indicate that the robot can communicate normally with the cloud through the target wireless communication module, and the abnormal communication results are used to indicate that the robot cannot communicate with the cloud through the target wireless communication module.

[0132] The notification operation includes a cloud notification operation and / or a local broadcast operation. The cloud notification operation is an operation in which the cloud notifies the guest, and the local broadcast operation is an operation in which the robot notifies the guest by voice.

[0133] Controlling the robot to perform a notification operation according to the communication detection result includes the following steps: responding to the communication detection result, controlling the robot to perform a cloud notification operation, and / or, responding to the communication detection result, controlling the robot to perform a local broadcast operation.

[0134] Cloud notification operations include cloud direct notification operations or cloud forwarding notification operations. Cloud direct notification operations are operations in which the robot directly triggers cloud notifications to guests. Cloud forwarding notification operations are operations in which the robot triggers cloud notifications to guests through available devices.

[0135] In response to the communication detection result, controlling the robot to perform a cloud notification operation includes the following steps: if the communication detection result is a normal communication result, controlling the target wireless communication module to perform a cloud direct connection notification operation for sending a user notification command to the cloud, so that the cloud sends an item arrival notification to the target device according to the user notification command; if the communication detection result is an abnormal communication result, controlling the candidate wireless communication module to perform a cloud forwarding notification operation for sending a user notification command to an available device, so that the available device triggers the cloud to send an item arrival notification to the target device according to the user notification command; the candidate wireless communication module is another wireless communication module of at least two types of wireless communication modules, and the available device is a device that can establish a communication connection with the robot and communicate with the cloud.

[0136] For example, the robot is configured with a 5G communication module, a Bluetooth module and a WI FI module. The robot communicates with the relay device through the Bluetooth module or the WI FI module, and the relay device communicates with the cloud (ie, the server). Alternatively, the robot communicates with the cloud through the 5G communication module.

[0137] If the target communication module is a 5G communication module, when the robot detects that the 5G communication module can communicate with the cloud normally, the robot controls the 5G communication module to send a user notification command to the cloud, and the cloud responds to the user notification command and sends an item arrival notification to the target device, where the target device is the telephone device or voice broadcast device in the guest room. The guest knows through the telephone device or voice broadcast device that the robot has delivered the item, and can go out to receive the item.

[0138] When the robot detects that the 5G communication module cannot communicate with the cloud, the robot can select the WI FI module as a candidate wireless communication module, and control the WI FI module to send a user notification command to the relay device. The relay device sends the user notification command to the cloud. The cloud responds to the user notification command and sends an item arrival notification to the target device.

[0139] In response to the communication detection result, controlling the robot to perform a local broadcast operation includes the following steps: in response to the communication detection result, obtaining environmental noise information and loudness-related information collected by the robot when it is at the target position, the loudness-related information is used to constrain the loudness of the robot when performing the local broadcast operation, determining the target broadcast loudness according to the environmental noise information and the loudness-related information, and controlling the robot to perform the local broadcast operation according to the target broadcast loudness. The embodiment of the present application can prevent environmental noise from covering the robot's voice broadcast loudness, thereby ensuring that the guest can hear the robot's voice broadcast, and can also prevent the robot's excessively loud voice from affecting other guests around, which is conducive to improving the user experience.

[0140] The environmental noise information is used to indicate the noise situation of the robot at the target position. The loudness-related information may be time information or scene information, etc. Determining the target broadcast loudness according to the environmental noise information and the loudness-related information includes the following steps: if the loudness-related information is time information, determining the maximum loudness corresponding to the current time information of the robot, determining the noise loudness according to the environmental noise information, and determining any loudness between the noise loudness and the maximum loudness as the target broadcast loudness.

[0141] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0142] As another aspect of the embodiment of the present application, the embodiment of the present application provides a notification device, wherein the notification device can be a software module, the software module includes a number of instructions, which are stored in a memory, and the processor can access the memory and call the instructions for execution to complete the notification method described in each of the above embodiments.

[0143] In some embodiments, the notification device may also be constructed by hardware devices, for example, the notification device may be constructed by one or more chips, and each chip may work in coordination with each other to complete the notification methods described in the above embodiments. For another example, the notification device may also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0144] The notification device 800 includes an information acquisition module 81, a scene determination module 82, a time adjustment module 83 and a notification operation module 84. The information acquisition module 81 is used to acquire environmental information collected by the robot during the delivery process, the scene determination module 82 is used to determine the abnormal scene type according to the environmental information, the time adjustment module 83 is used to adjust the preset initial notification time according to the abnormal scene type to obtain the target notification time, and the notification operation module 84 is used to control the robot to go to the target location to perform the notification operation according to the target notification time.

[0145] The robot provided in the embodiment of the present application can reliably identify the abnormal scene type and adjust the initial notification time according to the abnormal scene type to perform the notification operation, thereby ensuring that the robot's arrival time and the customer's meal pick-up time can be coordinated and synchronized with each other to avoid guests waiting.

[0146] In some embodiments, the notification operation module 84 is specifically used to: detect whether the robot has reached a preset target position and whether the target notification time has arrived, obtain a time-space detection result, respond to the time-space detection result, and control the robot to perform a notification operation.

[0147] In some embodiments, the robot is configured with at least two types of wireless communication modules, and the notification operation module 84 is also specifically used to: respond to the time-space detection results, detect whether the target wireless communication module can communicate with the cloud, and obtain a communication detection result, wherein the target wireless communication module is one of the at least two types of wireless communication modules, and the robot is controlled to perform a notification operation according to the communication detection results.

[0148] In some embodiments, the notification operation includes a cloud notification operation and / or a local broadcast operation, and the notification operation module 84 is also specifically used to: respond to the communication detection result, control the robot to perform a cloud notification operation; and / or, respond to the communication detection result, control the robot to perform a local broadcast operation.

[0149] In some embodiments, the communication detection result includes a normal communication result or an abnormal communication result, and the cloud notification operation includes a cloud direct connection notification operation or a cloud forwarding notification operation. The notification operation module 84 is also specifically used to: if the communication detection result is a normal communication result, then control the target wireless communication module to execute a cloud direct connection notification operation regarding sending a user notification command to the cloud, so that the cloud sends an item arrival notification to the target device according to the user notification command; if the communication detection result is an abnormal communication result, then control the candidate wireless communication module to execute a cloud forwarding notification operation regarding sending a user notification command to an available device, so that the available device triggers the cloud to send an item arrival notification to the target device according to the user notification command; the candidate wireless communication module is another wireless communication module of at least two types of wireless communication modules, and the available device is a device that can establish a communication connection with the robot and communicate with the cloud.

[0150] In some embodiments, the notification operation module 84 is also specifically used to: respond to the communication detection results, obtain the environmental noise information and loudness related information collected when the robot is at the target position, the loudness related information is used to constrain the loudness of the robot when performing local broadcast operations, determine the target broadcast loudness according to the environmental noise information and loudness related information, and control the robot to perform local broadcast operations according to the target broadcast loudness.

[0151] In some embodiments, the notification operation module 84 is also specifically used for: if the time-space detection result is used to indicate that the robot has reached a preset target position and the target notification time has arrived, then the robot is controlled to perform a notification operation at the target notification time; if the time-space detection result is used to indicate that the robot has reached a preset target position and the target notification time has not arrived, then the arrival time of the robot at the target position is obtained, and the robot is controlled to perform a notification operation at the arrival time; if the time-space detection result is used to indicate that the robot has not reached the preset target position and the target notification time has arrived, then according to the preset time alarm threshold, the robot is controlled to perform a delivery abnormality notification operation; if the time-space detection result is used to indicate that the robot has not reached the preset target position and the target notification time has not arrived, then the step of returning to execute detection of whether the robot has reached the preset target position and whether the target notification time has arrived to obtain the time-space detection result and subsequent steps.

[0152] In some embodiments, the time adjustment module 83 is further specifically used to: determine the time deviation corresponding to the abnormal scenario type, and determine the target notification time according to the time deviation and the initial notification time.

[0153] It should be noted that the above notification device can execute the notification method provided in the implementation mode of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the implementation mode of the notification device, please refer to the notification method provided in the implementation mode of this application.

[0154] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which may be a microcontroller, a robot, or a server, etc. The electronic device 900 includes one or more processors 91 and a memory 92. The memory 92 is connected to the one or more processors 91, for example, connected to the processor 91 via a bus.

[0155] The processor 91 is configured to support the electronic device to perform the corresponding functions in the method in the above method embodiment. The processor can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field programmable gate array (FPGA), a general array logic (GAL) or any combination thereof.

[0156] The memory 92 is used to store program codes, etc. The memory may include a volatile memory (VM), such as a random access memory (RAM); the memory may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory may also include a combination of the above-mentioned types of memory.

[0157] The memory 92 can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the notification method in the embodiment of the present application. The processor executes various functional applications and data processing of the notification method and the notification device by running the non-volatile software programs, instructions and modules stored in the memory, that is, realizing the functions of the notification method and each module or unit of the notification device provided in the above method embodiment.

[0158] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. The data storage area may store data created according to the use of the notification device, etc. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the notification device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The one or more modules are stored in the memory, and when executed by the one or more processors, they execute the notification method in any of the above method embodiments, for example, execute the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0160] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0161] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0162] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A notification method, applied to a robot, characterized in that: include: Obtain environmental information collected by the robot during the delivery process; Determine the abnormal scene type according to the environmental information; Adjusting the preset initial notification time according to the abnormal scenario type to obtain a target notification time; The robot is controlled to go to a target location to perform a notification operation according to the target notification time.

2. The method according to claim 1, characterized in that The step of controlling the robot to go to a target location to perform a notification operation according to the target notification time includes: Detecting whether the robot has reached a preset target position and whether the target notification time has arrived, and obtaining a spatiotemporal detection result; In response to the spatiotemporal detection result, the robot is controlled to perform a notification operation.

3. The method according to claim 2, characterized in that The robot is equipped with at least two types of wireless communication modules; The step of controlling the robot to perform a notification operation in response to the spatiotemporal detection result includes: In response to the spatiotemporal detection result, detecting whether the target wireless communication module can communicate with the cloud, and obtaining a communication detection result, wherein the target wireless communication module is one of the at least two types of wireless communication modules; The robot is controlled to perform a notification operation according to the communication detection result.

4. The method according to claim 3, characterized in that The notification operation includes a cloud notification operation and / or a local broadcast operation; The controlling the robot to perform a notification operation according to the communication detection result comprises: In response to the communication detection result, controlling the robot to perform a cloud notification operation; and / or, In response to the communication detection result, the robot is controlled to perform a local broadcast operation.

5. The method according to claim 4, characterized in that The communication detection result includes a normal communication result or an abnormal communication result, and the cloud notification operation includes a cloud direct connection notification operation or a cloud forwarding notification operation; In response to the communication detection result, controlling the robot to perform a cloud notification operation includes: If the communication detection result is a normal communication result, controlling the target wireless communication module to execute a cloud direct connection notification operation for sending a user notification command to the cloud, so that the cloud sends an item arrival notification to the target device according to the user notification command; If the communication detection result is a communication abnormality result, the candidate wireless communication module is controlled to execute a cloud forwarding notification operation on sending a user notification command to an available device, so that the available device triggers the cloud to send an item arrival notification to the target device according to the user notification command. The candidate wireless communication module is another wireless communication module of at least two types of wireless communication modules, and the available device is a device that can establish a communication connection with the robot and communicate with the cloud.

6. The method according to claim 4, characterized in that In response to the communication detection result, controlling the robot to perform a local broadcast operation includes: In response to the communication detection result, obtaining environmental noise information and loudness-related information collected by the robot when the robot is at the target position, wherein the loudness-related information is used to constrain the loudness of the robot when performing a local broadcast operation; Determine a target broadcast loudness according to the environmental noise information and the loudness association information; The robot is controlled to perform a local broadcast operation according to the target broadcast loudness.

7. The method according to claim 2, characterized in that The step of controlling the robot to perform a notification operation in response to the spatiotemporal detection result includes: If the spatiotemporal detection result is used to indicate that the robot has reached a preset target position and the target notification time has arrived, controlling the robot to perform a notification operation at the target notification time; If the spatiotemporal detection result is used to indicate that the robot has reached a preset target position, and the target notification time has not arrived, then obtaining the arrival time of the robot at the target position, and controlling the robot to perform a notification operation at the arrival time; If the spatiotemporal detection result indicates that the robot has not reached the preset target location, and the target notification time has arrived, the robot is controlled to perform a delivery abnormality notification operation according to a preset time alarm threshold; If the spatiotemporal detection result is used to indicate that the robot has not reached the preset target position and the target notification time has not arrived, then return to execute the step of detecting whether the robot has reached the preset target position and whether the target notification time has arrived to obtain the spatiotemporal detection result and subsequent steps.

8. The method according to any one of claims 1 to 7, characterized in that: The adjusting the preset initial notification time according to the abnormal scenario type to obtain the target notification time includes: Determine the time deviation corresponding to the abnormal scene type; A target notification time is determined according to the time deviation and the initial notification time.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory is connected to the processor, the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the electronic device implements the notification method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the notification method according to any one of claims 1 to 8.

Images