A method and system for positioning indoor functional areas of a mobile robot

Abstract

The present application discloses a method for positioning a mobile robot in an indoor functional area, the method comprising: the mobile robot triggers repositioning, obtains current first feature information, based on the obtained first feature information, wherein the first feature information at least includes feature information output by at least one intelligent device in each functional area in response to repositioning; identifies pre-stored second feature information related to positioning the functional area, obtains functional area information according to the functional area associated with the second feature information, and uses the obtained functional area information as the current position information. The present invention can adopt a flexible positioning method according to the different functions of the mobile robot and the intelligent device, does not require additional modification of the external environment, has strong adaptability, and the combination of images and audio further improves the robustness of positioning, is low-cost, and provides an economical implementation method for indoor positioning in non-industrial environments.

Description

Technical Field

[0001] The present invention relates to the field of indoor positioning, and in particular to a positioning method and system for an indoor functional area of ​​a mobile robot. Background Art

[0002] In recent years, mobile robots have been widely used due to their high efficiency, flexibility, reliability, and suitability for harsh working environments. Currently, various indoor positioning methods are affected by factors such as cost, positioning accuracy, reliability, and ease of use. Generally, there are the following technical solutions:

[0003] 1. Based on LiDAR technology

[0004] By installing a single-line or multi-line laser radar device on the mobile robot equipment, a map of the surrounding environment is constructed in advance and saved as a known map; the map features of the surrounding environment are observed at any time (in real time) and compared with the features in the known map to locate the position of the robot itself in the map.

[0005] However, the positioning technology based on lidar also has obvious defects. If it encounters the "kidnapping problem" (manually moving a normally moving robot to another area), or if there are multiple small areas with similar features in the area where the mobile robot is located, the positioning of the mobile robot will fail.

[0006] 2. Based on visual technology

[0007] Computer vision analysis technology is used to analyze the characteristics of the surrounding environment, so as to construct a map and locate the surrounding environment. The disadvantage of using visual technology for positioning is that if the environment often undergoes many small changes, such as changes in the position of tables and chairs in the home, curtain openings and closings, changes in natural light and lighting, etc., there is a high probability that vision-based positioning will fail.

[0008] 3. Based on ultrasonic technology

[0009] Ultrasonic technology requires the placement of sound wave signal sources and receiving devices on the mobile robot and its environment. Due to the significant impact of multipath effects and non-line-of-sight propagation, indoor positioning either requires large hardware investment or has poor positioning results.

[0010] 4. Based on wireless communication technology

[0011] Indoor positioning solutions based on wireless communication technologies such as Bluetooth, Zigbee, and ultra-wideband (UWB) require the installation of a large number of signal transmitters and receivers on the mobile robot and its surrounding environment, which results in high implementation costs.

[0012] Positioning technology based on WiFi technology relies on the relative stability of WiFi signals in the environment. Once the WiFi signal is unstable, such as when the home wireless router is turned off or moved, positioning will fail or error.

[0013] 5. Based on RFID wireless radio frequency technology

[0014] Wireless radio frequency technology does not have communication capabilities and has poor anti-interference capabilities.

[0015] Another mobile robot positioning method based on infrared road signs uses infrared emitting diodes to make dot-matrix road signs and attaches them to the indoor ceiling. The mobile robot recognizes such infrared road signs to achieve positioning. This positioning method requires additional infrared road signs, which destroy the beauty of the indoor environment and is only suitable for applications in industrial production environments.

[0016] Among the above-mentioned positioning technologies, whether it is the lidar technology or vision technology based on feature comparison in the environment map, or the technology based on positioning through communication based on sending and receiving signals, they are basically designed for mobile robot positioning in industrial production environments. They are costly, have great changes to the application environment, and are not user-friendly. Summary of the invention

[0017] The present invention provides a method for positioning an indoor functional area of ​​a mobile robot, so as to solve the indoor positioning of the mobile robot in a non-industrial environment and improve the friendliness to the application environment.

[0018] The present invention provides a method for positioning a mobile robot's indoor functional area, the method comprising:

[0019] The mobile robot triggers relocation and obtains current first feature information, wherein the first feature information at least includes feature information output by at least one smart device in each functional area in response to relocation;

[0020] Based on the acquired first characteristic information, second characteristic information related to the positioning functional area stored in advance is identified,

[0021] The function area information is acquired according to the function area associated with the second characteristic information, and the acquired function area information is used as the current position information.

[0022] Preferably, the method further comprises,

[0023] The mobile robot performs initialization configuration, configures the positioning strategy according to the user's selection, and saves it; wherein the positioning strategy includes an image positioning method and / or an audio positioning method;

[0024] The mobile robot triggers relocation, and obtaining the current first feature information further includes:

[0025] Get the positioning strategy.

[0026] If the positioning strategy is based on image positioning, image-based positioning is triggered, and the current first feature information is obtained as image information within the field of view of the mobile robot, and / or image information output by the intelligent device;

[0027] If the positioning strategy is based on the audio signal positioning method, the positioning based on the audio signal is triggered, and the first audio signal output by each intelligent device respectively recorded by the mobile robot and the current first feature information is obtained;

[0028] If the positioning strategy is based on image and audio signal positioning, the audio signal-based positioning and image signal-based positioning are triggered in parallel to obtain the current first feature information, which is the image information within the field of view of the mobile robot and the first audio signals output by each intelligent device recorded by the mobile robot.

[0029] The positioning strategy also includes priorities set for the image positioning method and the audio positioning method, respectively.

[0030] The acquisition positioning strategy further includes triggering the execution of a first priority positioning method in order of priority; if the first priority positioning method fails, triggering a second priority positioning method; if the second priority positioning method fails, alternately executing the first priority and second priority positioning methods until positioning is successful.

[0031] Preferably, the image-based positioning further includes the mobile robot performing initialization configuration, collecting an image of the appearance of at least one marker in each functional area from the patrol perspective of the mobile robot;

[0032] The second feature information is at least one marker image information in the functional area,

[0033] The mobile robot triggers relocation and obtains the current first feature information, including: when the mobile robot triggers relocation, the mobile robot collects images within the current field of view; sending a relocation instruction to an intelligent device with an image acquisition function through a local area network, so that the intelligent device obtains image information within its current field of view in response to the relocation instruction;

[0034] The identifying, based on the acquired first feature information, the pre-stored second feature information related to the positioning function area includes:

[0035] Based on the image information within the current field of view collected by the mobile robot and / or the image information output by the intelligent device, analyzing and identifying whether there are markers for determining the functional area in the current image;

[0036] If yes, the step of acquiring the functional area information according to the functional area associated with the second feature information and using the acquired functional area information as the current location information is performed.

[0037] If not, collect the image within the next field of view;

[0038] The acquiring of functional area information according to the functional area associated with the second characteristic information and using the acquired functional area information as the current position information includes: determining the current functional area according to the functional area information corresponding to the identified marker.

[0039] The image-based positioning further includes training the image recognition model based on the marker image collected during the initialization process, and saving the functional area information corresponding to the training recognition result.

[0040] The analyzing and identifying whether the current image contains a marker for determining the functional area further includes analyzing and identifying whether the current image contains a marker for determining the functional area based on a trained image recognition model.

[0041] The positioning based on audio signals further includes: the mobile robot performs initialization configuration, records a first audio signal output by at least one smart device near each smart device, uses the recorded first audio signal as a standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device;

[0042] The second characteristic information is a standard audio signal stored during initialization;

[0043] The mobile robot triggers relocation, and obtaining the current first feature information includes: when the mobile robot triggers relocation, sending a relocation instruction to the smart device in the local area network through the local area network; recording the first audio signal currently outputted from each smart device in sequence,

[0044] The identifying, based on the acquired first characteristic information, the pre-stored second characteristic information related to the positioning function area comprises: matching the currently recorded first audio signal with the standard audio signal stored during initialization,

[0045] If the match is successful, the step of acquiring the functional area information according to the functional area associated with the second feature information and using the acquired functional area information as the current location information is executed;

[0046] If the matching is unsuccessful, the mobile robot performs positioning while moving;

[0047] The step of obtaining functional area information according to the functional area associated with the second characteristic information and using the obtained functional area information as the current position information includes obtaining the current functional area information according to the correspondence between the standard audio signal and the functional area information and / or the smart device.

[0048] Wherein, the positioning based on audio signals and the positioning based on audio signals further include:

[0049] The mobile robot performs initialization configuration, and the mobile robot records the audio signal output by at least one intelligent device in each functional area, uses the audio signal as a standard audio signal to identify the functional area where the mobile robot is located, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the intelligent device; and the mobile robot patrols and collects the appearance image of at least one marker in each functional area;

[0050] The second characteristic information is a standard audio signal stored during initialization, and at least one marker image information in a functional area;

[0051] The mobile robot triggers relocation and obtains the current first feature information, comprising: when the mobile robot triggers relocation, the mobile robot sends a relocation instruction to each smart device through the local area network, and collects images within the current field of view;

[0052] The method of identifying the pre-stored second feature information related to the positioning of the functional area based on the acquired first feature information includes: analyzing and identifying whether there is a marker for determining the functional area in the current image based on the image information within the current field of view, and matching the currently recorded first audio signal with the standard audio signal stored during initialization;

[0053] If the image within the current field of view has the marker, or the currently recorded first audio signal successfully matches the standard audio signal stored during initialization, the step of acquiring the functional area information according to the functional area associated with the second feature information and using the acquired functional area information as the current position information is executed,

[0054] Otherwise, the image within the next field of view is collected and positioned while moving;

[0055] The step of obtaining functional area information according to the functional area associated with the second characteristic information and using the obtained functional area information as the current location information includes: obtaining the current functional area information according to the functional area information corresponding to the identified landmark, or according to the correspondence between the standard audio signal and the functional area information and / or the smart device.

[0056] Wherein, the mobile robot positioning during movement includes:

[0057] comparing the difference between the currently recorded first audio signal and each standard audio signal;

[0058] The functional area to which the standard audio signal corresponding to the minimum difference belongs is used as the moving direction for movement, and the smart device corresponding to the standard audio signal is used as the target smart device to be approached;

[0059] When the relocation trigger condition is met, a relocation instruction is sent to the target smart device, and a first audio signal currently output from the target device is recorded;

[0060] According to the increasing trend of the recorded current first audio signal, the moving direction is adjusted until the recorded current first audio signal matches the standard audio signal.

[0061] The difference is the difference between the volume value of the currently recorded first audio signal and the volume threshold of the standard audio signal.

[0062] The matching of the currently recorded first audio signal with the standard audio signal stored during initialization includes: matching the volume value of the currently recorded first audio signal with the volume threshold of the standard audio signal;

[0063] The increasing trend of the current first audio signal is the increasing trend of the volume value of the signal,

[0064] The relocation triggering condition includes that the movement displacement reaches a set displacement threshold, and / or the movement time reaches a set time threshold.

[0065] The present invention also provides a method for positioning a mobile robot's indoor functional area, the method comprising:

[0066] At least one intelligent device in each functional area receives a relocation instruction from the mobile robot.

[0067] Each smart device responds to the relocation instruction and obtains the current first feature information respectively.

[0068] Based on the acquired first characteristic information, identifying the pre-stored second characteristic information,

[0069] The functional area information is acquired according to the functional area associated with the second characteristic information, and the acquired functional area information is returned to the mobile robot as the current position information of the mobile robot.

[0070] Preferably, the method further comprises,

[0071] Receiving a positioning strategy from the mobile robot, wherein the positioning strategy includes an image positioning method and / or an audio positioning method;

[0072] Analyze the positioning strategy.

[0073] If the positioning strategy is based on image positioning, then respond to the repositioning instruction, trigger image-based positioning, and obtain the current first feature information as information of the image collected by the smart device.

[0074] If the positioning strategy is based on the audio signal positioning method, then respond to the repositioning instruction, trigger the positioning based on the audio signal, and obtain the current first feature information as the audio signal recorded by the smart device,

[0075] If the positioning strategy is based on image and audio signal positioning, then respond to the repositioning instruction, trigger image-based positioning and audio signal-based positioning in parallel, and obtain the current first feature information as information of the image captured by the smart device and the audio signal recorded by the smart device.

[0076] The positioning strategy also includes priorities set for the image positioning method and the audio positioning method, respectively.

[0077] The positioning strategy involved further includes triggering the execution of the first priority positioning method in order of priority; if the first priority positioning method fails, triggering the second priority positioning method; if the second priority positioning method fails, alternatingly executing the first priority and second priority positioning methods until the positioning is successful.

[0078] Wherein, the second characteristic information is the image information of the mobile robot;

[0079] The image-based positioning further includes performing initialization configuration, taking at least one functional area in each functional area where an intelligent device with an image acquisition function is located as attribute information of the image acquisition device, pre-storing image information of the mobile robot, or, based on the collected mobile robot image, training an image recognition model;

[0080] The obtaining of the current first feature information as information of the image collected by the smart device includes that each smart device collects image information in its current field of view respectively,

[0081] The identifying of the pre-stored second feature information based on the acquired first feature information includes analyzing the current image and identifying the mobile robot according to the pre-stored image information of the mobile robot or the trained image recognition model, and if the mobile robot is identified, executing the step of acquiring the functional area information according to the functional area associated with the second feature information,

[0082] The obtaining of the functional area information according to the functional area associated with the second characteristic information includes: the intelligent device that recognizes the image information of the mobile robot uses its functional area attribute information as the current position information of the mobile robot, and returns it to the mobile robot according to the IP address of the mobile robot.

[0083] Wherein, the smart device is an image acquisition device with a fixed lens and focal length,

[0084] The obtaining of the functional area information according to the functional area associated with the second feature information further includes obtaining the position of the mobile robot in the current functional area through coordinate conversion based on the position of the mobile robot in the image.

[0085] Preferably, the positioning based on audio signals further includes performing initialization configuration, wherein at least one smart device with an audio acquisition function in each functional area respectively records a first audio signal output by the mobile robot near the smart device, uses the recorded first audio signal as a standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device.

[0086] The second characteristic information is a standard audio signal stored during initialization configuration.

[0087] The identifying of the pre-stored second characteristic information based on the acquired first characteristic information includes that each smart device matches its currently recorded audio signal with the standard audio signal stored during the initialization configuration.

[0088] If the match is successful, the smart device whose standard audio signal is successfully matched executes the step of acquiring the functional area information corresponding to the smart device associated with the second feature information.

[0089] If the match is unsuccessful, the change trend of the current audio signal is monitored; wherein the change trend of the current audio signal is the change trend of the first audio signal output during the positioning process when the mobile robot moves in any direction without receiving any functional area information feedback from the smart device within a predetermined time threshold;

[0090] The obtaining of functional area information according to the functional area associated with the second characteristic information includes obtaining the functional area information according to the functional area information or smart device corresponding to the successfully matched standard audio signal; the smart device corresponding to the successfully matched standard audio signal returns the functional area information to the mobile robot according to the IP address of the mobile robot.

[0091] Preferably, the audio signal-based positioning and image-based positioning further include:

[0092] Performing initialization configuration, taking at least one functional area in each functional area where a smart device with an image acquisition function is located as attribute information of the smart device, pre-storing image information of the mobile robot, or training an image recognition model based on the collected mobile robot image; and / or

[0093] At least one smart device with an audio collection function in each functional area records the first audio signal output by the mobile robot near the smart device, uses the recorded first audio signal as the standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device.

[0094] The second characteristic information is the image information of the mobile robot and the standard audio signal stored during the initialization configuration;

[0095] The method of identifying the pre-stored second feature information based on the acquired first feature information includes: each intelligent device with an image acquisition function analyzes the current image respectively, and recognizes the image according to the pre-stored image information of the mobile robot or the trained image recognition model, and each intelligent device with an audio acquisition function matches its currently recorded audio signal with the standard audio signal stored during the initialization configuration.

[0096] If the mobile robot image is recognized, or the standard audio signal is matched successfully, the step of acquiring the functional area information according to the functional area associated with the second feature information is performed.

[0097] Otherwise, monitoring the change trend of the current audio signal; wherein the change trend of the current audio signal is the change trend of the first audio signal output during the positioning process when the mobile robot moves in any direction without receiving any functional area information feedback from the intelligent device within a predetermined time threshold;

[0098] The functional area information obtained according to the functional area associated with the second characteristic information includes: the intelligent device that recognizes the image information of the mobile robot uses its functional area attribute information as the current position information of the mobile robot, or obtains the functional area information according to the functional area information or intelligent device corresponding to the successfully matched standard audio signal; the intelligent device corresponding to the functional area returns the functional area information to the mobile robot according to the IP address of the mobile robot.

[0099] The matching of the currently recorded audio signal with the standard audio signal stored during the initialization configuration includes matching the volume value of the currently recorded audio signal with the volume threshold of the standard audio signal.

[0100] The monitoring of the changing trend of the current audio signal includes recording the changing trend of the volume value of the current audio signal, and the smart device whose volume value continues to increase sends an instruction to the mobile robot to maintain the current moving direction so that the mobile robot approaches the smart device.

[0101] The matching of the currently recorded audio signal with the standard audio signal stored during the initialization configuration includes matching the volume value of the currently recorded audio signal with the volume threshold of the standard audio signal.

[0102] The monitoring of the changing trend of the current audio signal includes, each smart device determining whether the mobile robot is moving away or approaching according to the Doppler effect of the audio signal and the change of the recorded frequency band of the first audio signal, adjusting the moving direction of the mobile robot so that the mobile robot continuously approaches the smart device with the largest volume value of the currently collected audio signal until the volume value recorded by the smart device matches the volume threshold of the standard audio signal corresponding to the smart device.

[0103] The present invention provides a system for realizing indoor functional area positioning of a mobile robot, the system comprising at least one mobile robot connected to a local area network, and at least one intelligent device connected to the local area network.

[0104] The mobile robot,

[0105] triggering relocation, obtaining current first characteristic information, wherein the first characteristic information at least includes characteristic information output by at least one smart device in each functional area in response to relocation;

[0106] Based on the acquired first characteristic information, second characteristic information related to the positioning functional area stored in advance is identified,

[0107] Acquire functional area information according to the functional area associated with the second characteristic information, and use the acquired functional area information as current position information;

[0108] The smart device,

[0109] Receive relocation instructions from the mobile robot,

[0110] In response to the relocation instruction, obtaining current third characteristic information,

[0111] Based on the acquired third characteristic information, identifying the pre-stored fourth characteristic information,

[0112] The functional area information is acquired according to the functional area associated with the fourth characteristic information, and the acquired functional area information is returned to the mobile robot as the current position information of the mobile robot.

[0113] Preferably, the system further comprises a gateway device that establishes a connection with the smart device and / or mobile robot via a local area network, and the gateway device transmits data for the smart device and / or mobile robot.

[0114] Preferably, the system further comprises a server connected to the gateway device via the Internet, wherein the server analyzes data from the smart device and / or the mobile robot and returns the analysis result.

[0115] The method for positioning the indoor functional area of ​​a mobile robot proposed in the present invention uses fixed objects (including intelligent devices) distributed in the functional area to locate the functional area where the mobile robot is located based on images and / or audio signals associated with the fixed objects. The present invention can adopt a flexible positioning method according to the different functions of the mobile robot and the intelligent device, does not require additional modification of the external environment, has strong adaptability, and the combination of images and audio further improves the robustness of positioning. It is low-cost and provides an economical implementation method for indoor positioning in non-industrial environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] Figure 1 It is a schematic diagram of the plane of smart home equipment.

[0117] Figure 2 A network topology diagram for smart home devices.

[0118] Figure 3 The present invention is a schematic diagram of enabling a mobile robot to obtain functional area information by using an image of the current mobile robot captured by an intelligent device with an image capture function.

[0119] Figure 4 A schematic diagram of the field of view of a smart device with a fixed lens.

[0120] Figure 5 A schematic diagram of a process for positioning a functional area for a mobile robot with image acquisition function.

[0121] Figure 6 The figure is a flowchart of positioning based on audio signals.

[0122] Figure 7 The figure is a flow chart of another embodiment of positioning based on audio signals.

[0123] Figure 8 The figure is a flow chart of the present invention using a combination of audio and image to perform positioning.

[0124] Fig. 9 A schematic diagram of a positioning method combining mobile audio and image in combination with Embodiment 1 and Embodiment 3 DETAILED DESCRIPTION

[0125] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0126] The applicant analyzed the consequences of the mobile robot's inability to locate itself in an indoor environment and found that: 1) it is unable to return to the charging station to automatically charge when the mobile robot's power is too low; 2) it is unable to go to a designated location to complete a specific task. Take a household mobile robot as an example. For example, the cleaning area of ​​a sweeping robot is divided into functional areas, such as the kitchen, living room, bedroom, etc. From the aforementioned point 2, the application environment does not require overly precise positioning, as long as it does not enter the wrong functional area. In addition, the current automatic recharging technical solutions can generally guarantee recharging in a simple environment (the same room). That is to say, in non-industrial production application environments, for example, in consumer user environments such as airports, stations, docks, hotels, shopping malls, homes, office buildings, libraries, etc., the functional requirements of the mobile robot determine that the positioning accuracy does not need to be particularly high, and it only needs to ensure that the range of the located functional area is accurate.

[0127] The method for positioning the indoor functional area of ​​a mobile robot proposed in the present application utilizes fixed objects distributed in the functional area to locate the functional area where the mobile robot is located based on images and / or audio signals associated with the fixed objects.

[0128] The following is an example of positioning a mobile robot in a smart home system.

[0129] In terms of application functions, the smart home system mainly includes security protection functions, home appliance automation functions, and multimedia entertainment functions; among them, the devices corresponding to the security protection function may include: cameras, electronic cat eyes, infrared alarms, water immersion sensors, smoke sensors, fingerprint locks, etc., home appliance automation functions include smart sockets, smart curtains, etc., and multimedia entertainment functions include children's companion robots, sports cameras, etc.

[0130] In terms of indoor functional areas, the smart home system includes kitchen smart system, bathroom smart system, access control smart system, bedroom smart system, dining room smart system, living room smart system, etc.

[0131] See also Figure 1 As shown, Figure 1This is a schematic diagram of the plane of smart home equipment. According to the division of home functional areas, smart devices are installed in each functional area. For example, the entrance area is equipped with a porch camera and a smart lock, the kitchen area is equipped with a smoke sensor, the living room area is equipped with a customer camera, the dining area is equipped with a dining camera, the bathroom is equipped with a water sensor, and the bedroom area is equipped with smart curtains. The above areas can also be equipped with smart sockets. As an automated operation function, the mobile robot can be located in any functional area. Users use application software to manage smart devices in the home environment. For fixed installed devices, the functional area where they are located is used as the inherent attribute information of the smart device for management.

[0132] See also Figure 2 As shown, Figure 2 This is a network topology diagram of smart home devices. Smart devices are connected to smart home cloud services through their own network modules or through gateways. For example, cameras and mobile robots are connected to cloud services through their own network modules, and smart locks, smart curtains, water sensors, smoke sensors, smart sockets, etc. are connected to cloud services, such as servers, through gateways.

[0133] Embodiment 1:

[0134] See also Figure 3 As shown, Figure 3 The present invention is a schematic diagram of enabling a mobile robot to obtain functional area information by using an image of the current mobile robot captured by an intelligent device (image acquisition device) having an image capture function.

[0135] During the initial configuration, a correspondence between the intelligent device and the functional area is established, that is, the functional area where the intelligent device is located is used as the attribute information of the intelligent device; the image information of the mobile robot is pre-stored for image analysis and recognition.

[0136] When the mobile robot encounters a "kidnapping problem" or a "lost problem" in a home environment or other situations that cause the mobile robot to be unable to correctly determine its own position, the mobile robot issues a relocation command, which is sent to the smart devices connected to the same local area network through the network (local area network) of smart home devices;

[0137] After receiving the repositioning instruction, the intelligent device with image acquisition function fixedly installed in the functional area triggers the analysis of the currently acquired image; if the mobile robot is identified from the image, it means that the mobile robot is currently in the field of view of the intelligent device, then the intelligent device returns its area attributes to the mobile robot, and the mobile robot determines the current area based on the area attributes, thereby completing the positioning.

[0138] For image acquisition devices with fixed lenses, fixed focal lengths, and no pan / tilt function, their field of view is also fixed. Figure 4 As shown, Figure 4 A schematic diagram of the field of view of an image acquisition device with a fixed lens, where the shaded area is the field of view of the living room camera. If a mobile robot is identified at a certain position in the image captured by the camera, the specific position of the mobile robot in the living room can be determined through coordinate conversion.

[0139] The analysis of the above images can be implemented using video algorithms, which can be run locally or on a cloud server.

[0140] The positioning method of this embodiment identifies the mobile robot based on the images collected by the image acquisition device, and is suitable for public places where a large number of image acquisition devices are installed, such as stations, airports, office buildings, etc.

[0141] Embodiment 2:

[0142] In actual applications, image acquisition equipment is usually not installed in functional areas involving privacy, such as bathrooms and bedrooms in a home environment. Example 2 provides a method for a mobile robot with image acquisition function to locate the functional area through the acquired feature images.

[0143] See also Figure 5 As shown, Figure 5 A schematic diagram of a process for positioning a functional area for a mobile robot with image acquisition function.

[0144] Step 501, when the mobile robot is initialized, the appearance image of at least one marker in each functional area is collected from the patrol perspective of the mobile robot, wherein the functional area marker may be a fixed object, for example, a fixedly installed smart device, a kitchen smoke sensor, a bathroom water sensor, etc. If the appearance of the smart devices is similar, electrical appliances such as televisions and refrigerators and / or large furniture that are less likely to move may also be used as markers.

[0145] Step 502: In order to improve the recognition accuracy of the image, preferably, the image recognition model is trained based on the acquired landmark image, and the functional area information corresponding to the training recognition result is saved locally or uploaded to the cloud server. The above image training and recognition process can be performed on the mobile robot or on the cloud server.

[0146] Step 503, when the mobile robot triggers repositioning, the mobile robot collects images within the current field of view, and based on the trained image recognition model, analyzes and identifies whether there are markers for determining functional areas in the current image. If so, the current functional area is determined based on the functional area information corresponding to the marker. If not, the mobile robot collects images within the next field of view, for example, adjusts the camera to another field of view adjacent to the current field of view until the marker is identified.

[0147] Furthermore, in view of the good correlation between smart devices and mobile robots in non-industrial indoor application scenarios, in combination with Example 1, the above step 503 also includes that the smart device with an image acquisition function obtains image information within its current field of view in response to the repositioning of the mobile robot, and sends the image information to the mobile robot, and the mobile robot performs image recognition based on the images collected by the mobile robot and / or the images collected by the smart device.

[0148] The mobile robot positioning method in this example is that the mobile robot collects external images by itself and / or the current images acquired by the intelligent device to complete the positioning, which improves the robustness of the system and the efficiency of image recognition, facilitates performance upgrades, has low requirements for equipment in the application environment, and has wide applicability.

[0149] Embodiment 3:

[0150] In order to prevent the positioning method of image acquisition and recognition from being limited by the influence of light, image acquisition conditions, etc., this embodiment proposes a positioning method based on audio signals.

[0151] See also Figure 6 As shown, Figure 6 This is a flowchart of positioning based on audio signals. In this embodiment, the fixedly installed intelligent device has an audio collection function, and the mobile robot has an audio signal output function. The intelligent device determines the current position of the mobile robot through the collected audio signals of a certain frequency band and a certain volume. The method specifically includes:

[0152] Step 601, initialization configuration: the mobile robot outputs a first audio signal of a certain frequency band and a certain volume near the location of each smart device. The frequency band can be a frequency band audible to humans or an ultrasonic frequency band. In terms of the content carried by the signal, the audio signal can be a voice or a piece of music, as long as it can be collected by the smart device.

[0153] The smart device records the frequency band and volume of the audio signal output by the mobile robot near the smart device, uses the recorded audio signal as the standard audio signal of the mobile robot in the functional area, and establishes a correspondence between the standard audio signal and the functional area information, and / or the smart device. For example, the functional area information of the smart device is used as the attribute information of the standard audio signal.

[0154] Step 602, when the mobile robot triggers repositioning, the mobile robot outputs the first audio signal. Since the smart devices distributed in various functional areas are at different distances from the mobile robot and are affected by partitions such as walls and furniture, the volume of audio signals collected by smart devices in different functional areas is different. Each smart device matches the currently collected audio signal with the volume threshold of the standard audio signal stored during initialization. Preferably, the threshold can be an interval value.

[0155] If the currently collected volume value is within the standard audio signal volume value threshold range, it is determined that the mobile robot is in the functional area where the smart device is located, and the smart device feeds back the functional area information corresponding to the standard audio signal to the mobile robot;

[0156] If the mobile robot does not receive any feedback from any smart device within a preset time threshold, the mobile robot will continue to output the first audio signal while moving in any direction, and each smart device will record the volume value change trend of the collected audio signal. The smart device whose volume value continues to increase will send a command to the mobile robot to maintain the current moving direction of the mobile robot, so that the mobile robot will approach the smart device until the volume value collected by the smart device is within the volume threshold range of the standard audio signal. The smart device will then feed back the functional area information corresponding to the recorded standard audio information to the mobile robot, so that the mobile robot can determine its current location. In the above process of approaching the smart device based on the volume value change trend, the moving direction of the mobile robot can be adjusted multiple times.

[0157] When the mobile robot moves in any direction, each intelligent device can also determine whether the mobile robot is moving away or approaching based on the Doppler effect of the audio signal and the changes in the frequency band of the collected first audio signal, and then adjust the moving direction of the mobile robot so that the mobile robot continues to approach the intelligent device with the largest volume value of the currently collected audio signal until the volume value measured by the intelligent device is within the volume threshold range of the standard audio signal.

[0158] In the above implementation, for different smart devices, the configured standard audio signals are the same, so as to reduce complexity.

[0159] As another implementation, different smart devices may be configured with different standard audio signals. For example, different smart device configurations correspond to audio signals of different frequency bands. In this way, the mobile robot outputs audio signals of all different frequency bands during the repositioning process. Each smart device has a different distance from the mobile robot and a different audio signal transmission path, so the strength of the received audio signal is different. Therefore, the smart device that receives the strongest audio signal returns the functional area attribute information of the smart device to the mobile robot, so that the mobile robot obtains the current functional area information. If no smart device returns the functional area attribute information, the moving direction of the mobile robot can be adjusted in the direction of the audio signal enhancement to approach the smart device that receives the strongest audio signal.

[0160] If there are multiple smart devices in a functional area, only one of them can be selected as the smart device marking the functional area during initial configuration to further reduce overall complexity.

[0161] The mobile robot positioning method in this embodiment is easy to integrate into various intelligent devices and mobile robots because the audio collection function and the audio output function are easy to implement. It is less affected by external factors and has strong anti-interference and robustness. The positioning method based on the mobile robot does not require additional environmental modification, which is convenient for upgrading the mobile robot.

[0162] Embodiment 4:

[0163] Based on the concept of Example 3, the audio collection function in the smart device and the audio playback function in the mobile robot can be interchanged, the smart device outputs the audio signal, the mobile robot records the collected audio signal of the smart device, and uses the audio signal as the standard audio signal.

[0164] See also Figure 7 As shown, Figure 7 A flowchart of another embodiment of positioning based on audio signals is shown. Specifically,

[0165] Step 701, initialization configuration: In the functional area where each smart device is located, the mobile robot collects and records the audio signal output by each smart device respectively, and the audio signal is used as a standard audio signal to identify each functional area, and a corresponding relationship between the standard audio signal and the functional area information and / or the smart device is established.

[0166] Step 702, when the mobile robot encounters a "kidnapping problem" or a "lost problem" or other situations that cause the mobile robot to be unable to correctly determine its own position, the mobile robot issues a relocation command and sends it to all smart devices in the same local area network through the network;

[0167] Step 703: After receiving the relocation instruction, all smart devices output audio signals in sequence. The mobile robot collects the current audio signal and matches the volume value of the current audio signal with the volume threshold range of each standard audio signal stored during initialization configuration.

[0168] If the match is successful, it is determined that the mobile robot is in the functional area corresponding to the matched standard audio signal, and the current functional area is obtained according to the corresponding relationship between the standard audio signal and the functional area information.

[0169] If the match is unsuccessful, the mobile robot compares the difference between the volume value of the current audio signal and the volume threshold of each standard audio signal. Since the larger the volume value of the current audio signal is, the closer it is to the volume threshold of a standard audio signal, and thus the smaller the difference is, the functional area to which the standard audio signal corresponding to the minimum difference belongs can be used as the direction of movement, and the smart device corresponding to the standard audio signal can be used as the target smart device to be approached;

[0170] When the moving displacement reaches the set displacement threshold, or the moving time reaches the set time threshold, the mobile robot sends a repositioning instruction to the target smart device according to the IP address of the target smart device; the target smart device responds to the repositioning instruction and outputs an audio signal; the mobile robot gradually adjusts the direction of movement according to the volume value of the current audio signal collected, until the volume value of the collected audio signal matches the volume threshold range of each standard audio signal stored during the initialization configuration, thereby obtaining the current functional area of ​​the mobile robot.

[0171] In this embodiment, if there are multiple smart devices in the functional area, only one of them can be selected as the smart device marking the functional area during initial configuration, so as to further reduce the overall complexity.

[0172] The positioning method of the mobile robot in this embodiment has low complexity, is easy to implement, is less affected by external factors, has strong anti-interference and robustness, and is suitable for private application environments where it is inconvenient to install image acquisition equipment.

[0173] Embodiment 5:

[0174] The mobile robot positioning method provided in this embodiment adopts a combination of audio and image to perform positioning, so as to further improve the robustness of positioning. In this embodiment, the intelligent device has an audio signal output function, and the mobile robot has an audio signal acquisition function and an image acquisition function.

[0175] See also Figure 8 As shown, Figure 8 This is a flow chart of positioning using a combination of audio and image in Embodiment 5 of the present invention.

[0176] Step 801, initialization configuration:

[0177] In the same functional area, the user selects at least one smart device as the smart device marking the functional area. If the reliability of positioning needs to be improved, multiple devices can be selected. The mobile robot respectively collects and records the audio signals output by the smart devices in each functional area. The audio signals are used as standard audio signals to identify each functional area, and a correspondence between the standard audio signals and the functional area information and / or smart devices is established.

[0178] In the same functional area, the mobile robot patrols and collects the appearance image of at least one marker in the functional area, trains the image recognition model based on the collected marker image, and saves the functional area information corresponding to the training and recognition result locally or uploads it to the cloud service end. The above-mentioned image training and recognition process can be executed on the mobile robot or on the cloud service end.

[0179] Furthermore, since the mobile robot can use audio signal positioning, image positioning, or both, the user can select different positioning methods according to the needs and form a positioning strategy to save as configuration information. In addition, the user can also set different priorities for different positioning methods.

[0180] Step 802: When the mobile robot triggers relocation, the mobile robot obtains a positioning strategy.

[0181] If the positioning strategy selected by the user is based on the audio signal positioning method, the mobile robot sends a repositioning instruction to each smart device through the local area network. Subsequently, the mobile robot and the smart device perform positioning according to step 703 in Example 4.

[0182] If the positioning strategy selected by the user is based on image positioning, the mobile robot performs positioning according to step 503 in embodiment 2.

[0183] If a priority is set in the positioning strategy selected by the user, for example, image positioning takes precedence over audio positioning, the mobile robot first performs positioning according to step 503 in Example 2; when image positioning fails, a repositioning instruction is sent to each smart device, and subsequently, the mobile robot and the smart device perform positioning according to step 703 in Example 4; when audio positioning fails, image positioning and audio positioning are performed alternately until positioning is completed.

[0184] If the positioning strategy selected by the user is audio and image positioning, but the priority is not set, in view of the fact that the audio positioning method is less affected by external factors, the mobile robot can be set to perform audio positioning by default first. When the audio positioning fails, the image positioning is used. The mobile robot first sends a repositioning instruction to each smart device through the local area network. Subsequently, the mobile robot and the smart device perform positioning according to step 703 in Example 4; when the audio positioning fails, the mobile robot triggers repositioning again, the mobile robot collects the image within the current field of view, and performs positioning according to step 503 in Example 2; when the image positioning fails, the audio positioning and image positioning are performed alternately until the positioning is completed. Another implementation method is that the audio and image positioning are performed in parallel, the mobile robot sends a repositioning instruction to each smart device, and collects the image within the current field of view at the same time. Step 703 in Example 4 and step 503 in the embodiment are performed in parallel. When the positioning result is obtained under any positioning method, the positioning is completed. This audio and image parallel positioning method is conducive to improving the efficiency and robustness of positioning.

[0185] The mobile robot positioning method of this embodiment can determine the positioning method that is most suitable for the external environment by combining and selecting different positioning methods in combination with actual application scenarios, which can not only meet the positioning efficiency, but also improve the positioning robustness and flexibility.

[0186] The positioning method combining audio and image adopted in this embodiment is a combination of Embodiment 2 and Embodiment 4. In specific applications, any one of Embodiments 1 and 2 can be combined with any one of Embodiments 3 and 4 according to the actual conditions of the mobile robot, intelligent device, and image acquisition device.

[0187] Embodiment 6:

[0188] join Fig. 9 As shown, Fig. 9 A schematic diagram of a positioning method combining mobile audio and image combined with Embodiment 1 and Embodiment 3. In this embodiment, at least one smart device has an audio signal output function, and at least one smart device has an image acquisition function, or at least one smart device has both an audio signal output function and an image acquisition function, and the mobile robot has an audio signal acquisition function and an image acquisition function.

[0189] Step 901, perform initialization configuration:

[0190] Taking at least one functional area in each functional area where a smart device with an image acquisition function is located as attribute information of the smart device, pre-storing image information of the mobile robot, or training an image recognition model based on the collected mobile robot image; and / or

[0191] At least one smart device with an audio collection function in each functional area records the first audio signal output by the mobile robot near the smart device, uses the recorded first audio signal as the standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device.

[0192] Step 902: The mobile robot triggers relocation, sends a relocation instruction to all smart devices through the local area network, and sends a positioning strategy set by the user; wherein the positioning strategy includes an image positioning method and / or an audio positioning method, and further, the priority of the two methods can be set.

[0193] Step 903: the smart device analyzes the positioning strategy.

[0194] If the positioning strategy is based on the image positioning method, then in response to the repositioning instruction, the image-based positioning method is triggered, that is, the step of triggering the analysis and recognition of the currently collected image in Example 1 is executed.

[0195] If the positioning strategy is based on the audio signal positioning method, then in response to the repositioning instruction, the positioning method based on the audio signal is triggered, that is, step 602 in embodiment 3 is executed.

[0196] If the positioning strategy is based on the image and audio signal positioning method, then in response to the repositioning instruction, the audio signal-based positioning method and the image-based positioning method are triggered in parallel, that is, the step of triggering the analysis and recognition of the currently collected image in embodiment 1 and executing step 602 in embodiment 3 are executed in parallel.

[0197] If it has a priority, the corresponding positioning method will be triggered in the order of priority. If the first priority positioning method fails, the second priority positioning method will be triggered. If the second priority positioning method fails, the first priority and second priority positioning methods will be executed alternately until the positioning is successful.

[0198] Although the functions of the smart devices themselves are different, it does not prevent the smart devices from processing the corresponding positioning methods. For example, when a smart device that does not have an image acquisition function receives a positioning instruction based on an image positioning method, the smart device does not need to respond to the repositioning instruction.

[0199] The embodiments of the present invention can take into account the positioning of mobile robots in the case of various intelligent devices with different functions in the same local area network, have strong adaptability and good flexibility, and can give full play to the role of various intelligent devices with different functions, thereby achieving higher robustness.

[0200] An embodiment of the present invention also provides a mobile robot that can realize indoor functional area positioning. The mobile robot includes a memory and a processor. The memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the positioning methods of embodiments 1-6.

[0201] An embodiment of the present invention also provides an intelligent device that can realize the positioning of indoor functional areas of a mobile robot. The intelligent device includes a memory and a processor. The memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement any positioning method described in Examples 1-6.

[0202] The embodiment of the present invention provides a system for realizing indoor functional area positioning of a mobile robot. In terms of hardware structure, the system includes at least one mobile robot connected to a local area network and at least one intelligent device connected to the local area network. The local area network is connected to a cloud service, such as a server, via the Internet.

[0203] The mobile robot includes a first memory and a first processor, the first memory is used to store computer programs; the first processor is used to execute the program stored in the memory to implement any positioning method described in embodiments 1-6;

[0204] The smart device includes a second memory and a second processor, the second memory is used to store computer programs; the second processor is used to execute the programs stored in the memory to implement any positioning method described in Examples 1-6.

[0205] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage function located away from the aforementioned processor.

[0206] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0207] The present invention provides a system for realizing indoor functional area positioning of a mobile robot. In terms of networking function, the system includes at least one mobile robot connected to a local area network and at least one intelligent device connected to the local area network.

[0208] The mobile robot,

[0209] triggering relocation, obtaining current first characteristic information, wherein the first characteristic information at least includes characteristic information output by at least one smart device in each functional area in response to relocation;

[0210] Based on the acquired first characteristic information, second characteristic information related to the positioning functional area stored in advance is identified,

[0211] According to the functional area associated with the second feature information, the functional area information is obtained, and the obtained functional area information is used as the current position information

[0212] The smart device,

[0213] Receive relocation instructions from the mobile robot,

[0214] In response to the relocation instruction, obtaining current third characteristic information,

[0215] Based on the acquired third characteristic information, identifying the pre-stored fourth characteristic information,

[0216] The functional area information is acquired according to the functional area associated with the fourth characteristic information, and the acquired functional area information is returned to the mobile robot as the current position information of the mobile robot.

[0217] The system also includes a gateway device that establishes a connection with the smart device and / or mobile robot through a local area network, and the gateway device transmits data for the smart device and / or mobile robot. In this embodiment, data analysis and processing are completed by the smart device and mobile robot themselves.

[0218] As another embodiment, the system further includes a server connected to the gateway device via the Internet, wherein the server analyzes the data from the smart device and / or the mobile robot and returns the analysis result.

[0219] It should be noted that the various embodiments of the present invention are combined and adjusted according to the functions of the smart devices and mobile robots, and the distribution and installation of smart devices in various functional areas. For example, during the mobile positioning process of the mobile robot, the direction of movement can be adjusted in combination with the semantic understanding of natural language.

[0220] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0221] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for positioning a mobile robot in an indoor functional area, characterized in that: The method includes, The mobile robot triggers relocation, triggering at least one of image-based positioning and audio signal-based positioning. In the case of image-based localization: Acquire current first feature information, wherein the first feature information includes: image information of a marker within the field of view of the mobile robot, and / or feature information output by at least one smart device in each functional area in response to repositioning from the mobile robot, the feature information including image information of the mobile robot, Based on the acquired first feature information, second feature information pre-stored and related to the positioning functional area is identified, wherein the second feature information includes: marker image information and / or mobile robot image information, Acquire functional area information according to the functional area associated with the second characteristic information, and use the acquired functional area information as current position information; In the case of audio signal based positioning: Acquire current first characteristic information, wherein the first characteristic information includes: first audio signals outputted by each smart device respectively, Matching the currently acquired first audio signal with the stored standard audio signal, If the match is successful, the current functional area information is obtained according to the corresponding relationship between the standard audio signal and the functional area information and / or the smart device.

2. The method according to claim 1, characterized in that The method further comprises, The mobile robot performs initialization configuration, configures the positioning strategy according to the user's selection, and saves it; wherein the positioning strategy includes: image positioning method and / or audio positioning method; The mobile robot triggering relocation further comprises: Get the positioning strategy. If the positioning strategy is based on image positioning, image-based positioning is triggered; If the positioning strategy is based on audio signal positioning, then audio signal-based positioning is triggered; If the positioning strategy is based on image and audio signal positioning, the positioning based on the audio signal and the positioning based on the image signal are triggered in parallel.

3. The method according to claim 2, characterized in that The positioning strategy also includes: priorities set for the image positioning mode and the audio positioning mode respectively; The acquisition positioning strategy further includes: According to the priority order, trigger the execution of the first priority positioning method, If the first priority positioning method fails, the second priority positioning method will be triggered. If the second priority positioning method fails, the first priority and second priority positioning methods are executed alternately until the positioning succeeds.

4. The method according to claim 3, characterized in that The image-based positioning further includes the mobile robot performing initial configuration, collecting an image of the appearance of at least one marker in each functional area from the patrol perspective of the mobile robot; In the case of image-based positioning, the obtaining of the current first feature information includes: When the mobile robot triggers relocation, the mobile robot collects images within the current field of view; sends a relocation instruction to an intelligent device with an image acquisition function through a local area network, so that the intelligent device obtains image information within its current field of view in response to the relocation instruction; The identifying, based on the acquired first feature information, pre-stored second feature information related to the positioning function area includes: Analyze and identify whether there are markers for determining the functional area in the current image based on the image information within the current field of view collected by the mobile robot, and / or Based on the image information output by the intelligent device, analyzing and identifying whether the current image contains the mobile robot; If yes, the step of acquiring the functional area information according to the functional area associated with the second feature information and using the acquired functional area information as the current location information is performed. If not, collect the image within the next field of view; The acquiring the functional area information according to the functional area associated with the second characteristic information and using the acquired functional area information as the current location information comprises: The current functional area is determined based on the functional area information corresponding to the identified markers and / or the mobile robot.

5. The method according to claim 4, characterized in that The image-based positioning further comprises: Based on the landmark images collected during the initialization process, the image recognition model is trained, and the functional area information corresponding to the training and recognition results is saved, and / or Based on the mobile robot images collected during the initialization process, the image recognition model is trained. The analyzing and identifying whether there is a marker for determining a functional area in the current image further includes: Based on the trained image recognition model, analyze and identify whether there are landmarks and / or mobile robots for determining functional areas in the current image.

6. The method according to claim 3, characterized in that The positioning based on the audio signal further comprises: The mobile robot performs initialization configuration, records a first audio signal output by at least one smart device near each smart device, uses the recorded first audio signal as a standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device; In the case of positioning based on an audio signal, the obtaining of the current first feature information includes: When the mobile robot triggers relocation, it sends a relocation instruction to the intelligent devices in the local area network through the local area network; and records the first audio signals currently outputted from each intelligent device in sequence; The matching of the currently acquired first audio signal with the stored standard audio signal further comprises: If the matching is unsuccessful, the mobile robot performs positioning while moving.

7. The method according to claim 2, characterized in that The method further comprises: In the case of positioning based on audio signals and positioning based on image signals, The mobile robot is initialized and configured. The mobile robot records the audio signal output by at least one intelligent device in each functional area, uses the audio signal as a standard audio signal for identifying the functional area where the mobile robot is located, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the intelligent device; and the mobile robot patrols and collects the appearance image of at least one marker in each functional area and / or trains the image recognition model based on the mobile robot image collected during the initialization process. When the mobile robot triggers relocation, it collects images within the current field of view, and sends a relocation instruction to an intelligent device with an image acquisition function through a local area network, so that the intelligent device obtains image information within its current field of view in response to the relocation instruction; Based on the image information within the current field of view collected by the mobile robot and / or the image information output by the intelligent device, analyzing and identifying whether there is a marker and / or a mobile robot for determining the functional area in the current image, and matching the currently recorded first audio signal with the standard audio signal stored during initialization; If the image within the current field of view has the marker and / or the mobile robot, or the currently recorded first audio signal successfully matches the standard audio signal stored during initialization, then the current functional area information is obtained according to the functional area information corresponding to the identified marker and / or mobile robot, or according to the corresponding relationship between the standard audio signal and the functional area information and / or the smart device, Otherwise, the image within the next field of view is collected and positioned during movement.

8. The method according to claim 6 or 7, characterized in that The mobile robot performs positioning during movement, including: comparing the difference between the currently recorded first audio signal and each standard audio signal; The functional area to which the standard audio signal corresponding to the minimum difference belongs is used as the moving direction for movement, and the smart device corresponding to the standard audio signal is used as the target smart device to be approached; When the relocation trigger condition is met, a relocation instruction is sent to the target smart device, and a first audio signal currently output from the target device is recorded; According to the increasing trend of the recorded current first audio signal, the moving direction is adjusted until the recorded current first audio signal matches the standard audio signal.

9. The method according to claim 8, characterized in that The difference is the difference between the volume value of the currently recorded first audio signal and the volume threshold of the standard audio signal, The matching of the currently recorded first audio signal with the standard audio signal stored during initialization includes: matching the volume value of the currently recorded first audio signal with the volume threshold of the standard audio signal; The increasing trend of the current first audio signal is the increasing trend of the volume value of the signal, The relocation triggering condition includes that the movement displacement reaches a set displacement threshold, and / or the movement time reaches a set time threshold.

10. A method for positioning a mobile robot in an indoor functional area, characterized in that: The method includes, At least one intelligent device in each functional area receives a relocation instruction from the mobile robot. Each smart device triggers at least one of image-based positioning and audio signal-based positioning in response to the repositioning instruction. In the case of image-based localization: Each smart device obtains current first feature information respectively, and the first feature information includes: each smart device collects image information of the mobile robot in its current field of view respectively, Each intelligent device analyzes the current image, and the intelligent device that recognizes the image information of the mobile robot uses the acquired functional area information as the current position information of the mobile robot and returns it to the mobile robot; In the case of audio-based positioning: Each smart device obtains current first feature information respectively, where the first feature information includes: an audio signal obtained by the smart device, Each smart device matches the acquired audio signal with the stored standard audio signal. If the match is successful, the smart device whose standard audio signal is successfully matched obtains its functional area information, and returns the obtained functional area information to the mobile robot as the current position information of the mobile robot.

11. The method according to claim 10, characterized in that The method further comprises, Receiving a positioning strategy from the mobile robot, wherein the positioning strategy includes an image positioning method and / or an audio positioning method; Analyze the positioning strategy. If the positioning strategy is based on image positioning, then responding to the repositioning instruction triggers image-based positioning. If the positioning strategy is based on audio signal positioning, then responding to the repositioning instruction, triggering positioning based on audio signals, If the positioning strategy is based on image and audio signal positioning, then in response to the repositioning instruction, image-based positioning and audio signal-based positioning are triggered in parallel.

12. The method according to claim 10, characterized in that The positioning strategy also includes priorities set for the image positioning mode and the audio positioning mode respectively. The acquisition positioning strategy involved further includes, According to the priority order, trigger the execution of the first priority positioning method, If the first priority positioning method fails, the second priority positioning method will be triggered. If the second priority positioning method fails, the first priority and second priority positioning methods are executed alternately until the positioning succeeds.

13. The method according to claim 12, characterized in that The image-based positioning further comprises: Performing initialization configuration, taking at least one functional area in each functional area where an intelligent device with an image acquisition function is located as attribute information of the image acquisition device, pre-storing image information of the mobile robot, or, based on the collected mobile robot image, training an image recognition model; The analysis of the current image by each smart device includes: The current image is analyzed and identified according to pre-stored image information of the mobile robot or a trained image recognition model. If the mobile robot is identified, the intelligent device that identifies the image information of the mobile robot uses its functional area attribute information as the current position information of the mobile robot and returns it to the mobile robot according to the IP address of the mobile robot.

14. The method according to claim 12, characterized in that The smart device is an image acquisition device with a fixed lens and focal length. The intelligent device that recognizes the image information of the mobile robot uses its functional area attribute information as the current position information of the mobile robot, and further includes obtaining the position of the mobile robot in the current functional area through coordinate conversion based on the position of the mobile robot in the image.

15. The method according to claim 12, characterized in that The positioning based on the audio signal further includes: Performing initialization configuration, at least one smart device with an audio acquisition function in each functional area respectively records the first audio signal output by the mobile robot near the smart device, using the recorded first audio signal as the standard audio signal of the mobile robot in the functional area, and establishing a corresponding relationship between the standard audio signal and the functional area information and / or the smart device; Each smart device matches the acquired audio signal with the stored standard audio signal, including: Each smart device matches its currently recorded audio signal with the standard audio signal stored during initial configuration. If the match is unsuccessful, the change trend of the current audio signal is monitored; wherein the change trend of the current audio signal is the change trend of the first audio signal output during the positioning process when the mobile robot moves in any direction without receiving any functional area information feedback from the smart device within a predetermined time threshold; The smart device that successfully matches the standard audio signal obtains its functional area information, and returns the obtained functional area information to the mobile robot as the current position information of the mobile robot, including: According to the functional area information or smart device corresponding to the successfully matched standard audio signal, the functional area information is obtained. The smart device corresponding to the successfully matched standard audio signal returns the functional area information to the mobile robot according to the IP address of the mobile robot.

16. The method according to claim 12, characterized in that The method further comprises, In the case of audio signal-based positioning and image-based positioning, Performing initialization configuration, taking at least one functional area in each functional area where a smart device with an image acquisition function is located as attribute information of the smart device, pre-storing image information of the mobile robot, or, based on the collected mobile robot image, training an image recognition model; and / or At least one smart device with an audio collection function in each functional area records the first audio signal output by the mobile robot near the smart device, uses the recorded first audio signal as the standard audio signal of the mobile robot in the functional area, and establishes a corresponding relationship between the standard audio signal and the functional area information and / or the smart device. Each intelligent device with an image acquisition function analyzes the current image and recognizes it according to the pre-stored image information of the mobile robot or the trained image recognition model, and each intelligent device with an audio acquisition function matches its currently recorded audio signal with the standard audio signal stored during the initialization configuration. If the image of the mobile robot is recognized, or the standard audio signal is successfully matched, the intelligent device that recognizes the image information of the mobile robot uses its functional area attribute information as the current position information of the mobile robot, or obtains the functional area information according to the functional area information or intelligent device corresponding to the successfully matched standard audio signal; the intelligent device corresponding to the functional area returns the functional area information to the mobile robot according to the IP address of the mobile robot. Otherwise, monitor the changing trend of the current audio signal; wherein the changing trend of the current audio signal is the changing trend of the first audio signal output during the positioning process when the mobile robot moves in any direction without receiving any functional area information feedback from the intelligent device within a predetermined time threshold.

17. The method according to claim 15 or 16, characterized in that The matching of the currently recorded audio signal with the standard audio signal stored during the initialization configuration includes matching the volume value of the currently recorded audio signal with the volume threshold of the standard audio signal, The monitoring of the changing trend of the current audio signal includes recording the changing trend of the volume value of the current audio signal, and the smart device whose volume value continues to increase sends an instruction to the mobile robot to maintain the current moving direction so that the mobile robot approaches the smart device.

18. The method according to claim 15 or 16, characterized in that The matching of the currently recorded audio signal with the standard audio signal stored during the initialization configuration includes matching the volume value of the currently recorded audio signal with the volume threshold of the standard audio signal, The monitoring of the changing trend of the current audio signal includes, each smart device determining whether the mobile robot is moving away or approaching according to the Doppler effect of the audio signal and the change of the recorded frequency band of the first audio signal, adjusting the moving direction of the mobile robot so that the mobile robot continuously approaches the smart device with the largest volume value of the currently collected audio signal until the volume value recorded by the smart device matches the volume threshold of the standard audio signal corresponding to the smart device.

19. A system for realizing indoor functional area positioning of a mobile robot, characterized in that: The system includes at least one mobile robot connected to a local area network and at least one intelligent device connected to the local area network. The mobile robot is configured to perform the steps of the method for positioning the indoor functional area of ​​the mobile robot as claimed in any one of claims 1 to 9 The smart device is configured to execute the steps of the method for positioning the indoor functional area of ​​a mobile robot as claimed in any one of claims 10 to 18.

20. The system of claim 19, wherein: The system also includes a gateway device that establishes a connection with the smart device and / or the mobile robot via a local area network, and the gateway device transmits data for the smart device and / or the mobile robot.

21. The system according to claim 19 or 20, characterized in that The system also includes a server connected to the gateway device via the Internet, wherein the server analyzes data from the smart device and / or the mobile robot and returns the analysis result.

Images