Obstacle avoidance prompting method, electronic equipment and computer readable medium

Abstract

The invention provides an obstacle avoidance prompting method. The method comprises the following steps: receiving a road section image of a target road section collected by target camera equipment; determining obstacle information of the target road section according to the road section image; the obstacle information of the target road section represents the obstacle condition on the target road section; in response to the obstacle on the target road section, generating corresponding prompt information and sending the prompt information to the audio output equipment; the prompt information is used for the audio output device to play the obstacle avoidance information voice. The invention further provides electronic equipment and a computer readable medium.

Description

Technical Field

[0001] The present disclosure relates to the technical field of assisted obstacle avoidance, and in particular to an obstacle avoidance prompting method, an electronic device and a computer-readable medium. Background Art

[0002] Most of the existing guidance devices for visually impaired people (such as blind people) are individual devices with road recognition and identification functions to guide the blind. However, the cost of individual devices with road recognition and identification functions is high, which is expensive for individuals, inconvenient to use, and different individual devices are not universal. Summary of the invention

[0003] The present disclosure aims to solve at least one of the technical problems existing in the prior art, and proposes an obstacle avoidance prompt method, an electronic device and a computer-readable medium.

[0004] To achieve the above objectives, in a first aspect, an embodiment of the present disclosure provides an obstacle avoidance prompting method for an IoT platform, comprising:

[0005] Receiving a road section image of a target road section captured by a target camera device;

[0006] Determine obstacle information of the target road section according to the road section image; the obstacle information of the target road section represents the situation of obstacles on the target road section;

[0007] In response to the existence of the obstacle on the target road section, corresponding prompt information is generated and sent to the audio output device; the prompt information is used for the audio output device to play the obstacle avoidance information voice.

[0008] In a second aspect, an embodiment of the present disclosure provides an obstacle avoidance prompt method for an audio output device, comprising:

[0009] Receive prompt information sent by the IoT platform;

[0010] According to the prompt information, the corresponding obstacle avoidance information voice is played; the obstacle avoidance information voice is used to instruct the target user to avoid obstacles.

[0011] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0012] one or more processors;

[0013] A memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle avoidance prompt method as described in the above embodiment.

[0014] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the obstacle avoidance prompt method as described in the above embodiment is implemented.

[0015] The present disclosure has the following beneficial effects:

[0016] The disclosed embodiments provide an obstacle avoidance prompt method, electronic device, and computer-readable medium. The target user with visual impairment can carry an audio output device, and the camera device can collect the road segment images of each road segment in real time. When the IoT platform recognizes that the target user is moving in the road segment image, the target user can be given accurate voice guidance to avoid dangerous objects. The camera device is a public IoT device, not a personal device, and has strong versatility. The visually impaired person can use the audio output device to guide the obstacle avoidance, which is low in cost and does not require the visually impaired person to operate the audio output device, so it is more convenient to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of an application scenario of an obstacle avoidance prompt method provided by an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of an obstacle avoidance prompt method for an IoT platform provided in an embodiment of the present disclosure;

[0019] Figure 3 This is a flowchart of a specific implementation method of step S23 in the embodiment of the present disclosure;

[0020] Figure 4 A flowchart of an obstacle avoidance prompt method for an audio output device provided in an embodiment of the present disclosure;

[0021] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0022] Figure 6 A schematic diagram of the structure of a computer-readable medium provided in an embodiment of the present disclosure;

[0023] Figure 7 A schematic diagram of the structure of an obstacle avoidance prompt system provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the control method of the control provided by the present disclosure, the electronic device and the computer-readable medium are described in detail below with reference to the accompanying drawings.

[0025] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0026] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0027] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0028] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0030] When visually impaired people (such as the blind) walk on the road, they cannot clearly see obstacles on the road, which can easily lead to danger.

[0031] Therefore, in some related technologies, the visually impaired person carries a personal obstacle avoidance prompt device to analyze obstacles in the direction of travel and prompt the person to avoid obstacles. That is, the obstacle avoidance prompt device needs to have a camera, which can collect images of the direction in which the visually impaired person is moving, analyze obstacles, and prompt the visually impaired person through voice.

[0032] However, the individual obstacle avoidance equipment for the visually impaired needs to be able to quickly and clearly capture images of the forward direction during movement, and the images need to be analyzed and calculated for obstacles, so the cost of the individual obstacle avoidance equipment is relatively high. Moreover, the individual obstacle avoidance equipment needs to be facing the front of the visually impaired to capture images during use. For the visually impaired, it is difficult to adjust the correct shooting direction when they cannot see the real-time shooting image of the camera. Therefore, it is inconvenient for the visually impaired to use the individual obstacle avoidance equipment.

[0033] Figure 1A schematic diagram of an application scenario of an obstacle avoidance prompt method provided by an embodiment of the present disclosure. Figure 1 Describe an exemplary application scenario of an obstacle avoidance prompt method:

[0034] Reference Figure 1 The application scenario is wireless networking, which includes: camera equipment, IoT platform, and audio output equipment.

[0035] Among them, the IoT platform is deployed in the computer room, has data processing capabilities, and can communicate with IoT devices connected to it, such as IoT management devices.

[0036] Camera equipment is a device that can capture images and is deployed in public areas where people walk (such as various surveillance cameras on the road). The camera equipment can transmit pictures, videos and other information to the IoT platform through the network.

[0037] The audio output device is a device (such as a smart headset) that the target user (visually impaired person) carries with him / her and can output audio. The audio output device can be connected to the IoT platform through smart wearable devices and the like. The embodiments of the present disclosure do not specifically limit the audio output device, which can be a headset, a mobile phone, etc. At the same time, the embodiments of the present disclosure do not specifically limit the interaction method between the audio output device and the IoT platform, which can be a WiFi (Wireless Fidelity, wireless local area network) network, a cellular network, Bluetooth communication, etc.

[0038] When the target user needs manual assistance, the IoT platform can trigger a real-time video forwarding process to provide manual assistance to the target user who is carrying an audio output device.

[0039] In some embodiments, the wireless networking may further include at least one associated device (eg, a first associated device and a second associated device), and the associated device may play the video according to instructions of the IoT platform.

[0040] Figure 2 A flowchart of an obstacle avoidance prompt method for an IoT platform provided in an embodiment of the present disclosure. Figure 2 The present disclosure provides an obstacle avoidance prompting method for an IoT platform, including:

[0041] S21, receiving a road section image of a target road section collected by a target camera device;

[0042] S22, determining obstacle information of the target road section according to the road section image; the obstacle information of the target road section represents the situation of obstacles on the target road section;

[0043] S23. In response to the obstacle existing on the target road section, generate corresponding prompt information and send it to an audio output device; the prompt information is used for the audio output device to play obstacle avoidance information voice.

[0044] In the embodiment of the present disclosure, the road section image of the target road section is collected by the target camera device, and the target camera device has the function of capturing and real-time video transmission. The embodiment of the present disclosure does not limit the number of target camera devices, which can be one or more. In some embodiments, the target camera device is set in a public area space, and is installed in a fixed target road section in the public area space, and the road section image of the fixed target road section can be transmitted to the IoT platform in real time. In some related technologies, the target user is navigated by a GIS (Geographic Information System) system. While using the IoT platform for navigation, the IoT platform is used to analyze obstacles (including fixed obstacles, such as electric poles, mailboxes, etc., such as obstacles that may move, no vehicles, pedestrians, etc.) in the road section image in real time. When there are obstacles on the target road section, the obstacle information is converted into prompt information, thereby guiding the target user to avoid obstacles, providing obstacle avoidance guidance for the target user, and effectively improving the accuracy of navigation.

[0045] The IoT platform has functions such as target camera equipment management, road section image analysis, voice conversion, and real-time video processing. It can display the real-time video of the target road section where the target user who is currently reporting the obstacle is located, and manage the target camera equipment in a unified manner. In some embodiments, the IoT platform receives the road section image of the target road section collected by the target camera equipment, and analyzes the road condition of the target road section through the road section image to determine the obstacle information of the target road section. When the obstacle exists on the target road section, the road condition is converted through the intelligent voice conversion system, and the corresponding prompt information is generated and sent to the audio output device, so that the target user carrying the audio output device can interactively broadcast the obstacle avoidance information voice of the current target road section, and can actively remind the visually impaired target user to avoid obstacles.

[0046] The audio output device is connected to the IoT platform and can interact with the IoT platform by voice. In other embodiments, when it is detected that the target user encounters other problems to be solved besides obstacles, the IoT platform can also assist the target user in real time through the road section image of the target road section collected by the target camera device and the audio output device, so as to further achieve the purpose of helping the target user and provide care for the visually impaired target user.

[0047] In addition, in the embodiments of the present disclosure, there is no special restriction on the function of the camera device, and in addition to shooting the road section image of the target road section, it can also have an audio output function, etc. On this basis, it should be understood that in the embodiments of the present disclosure, the prompt information can also be sent to the camera device, and the camera device plays the obstacle avoidance information voice on the target road section to achieve guidance for the target user.

[0048] In some embodiments, before S23, the method further includes:

[0049] Get the current position of the audio output device;

[0050] The road section where the current position is located is determined as the target road section, and the camera device that can capture an image of the target road section is determined as the target camera device.

[0051] It should be understood that in the embodiments of the present disclosure, the IoT platform can determine the target camera device corresponding to the current position of the audio output device based on positioning search and other methods. In some embodiments, the target camera device is closest to the current position and can capture the image corresponding to the current position.

[0052] In other embodiments, the audio output device is carried by the target user, and the audio output device can not only output audio but also locate in real time, that is, it can report the current position of the audio output device to the Internet of Things platform. The road section where the current position of the audio output device is located is the target road section, and the camera device that can capture images of the target road section is the target camera device.

[0053] In some embodiments, S21 includes:

[0054] When the current position is continuously located in the target road section, the road section image of the target road section captured by the target camera device is periodically received.

[0055] In an embodiment of the present disclosure, a camera device is fixedly installed at a target section, and the road section image of the target section is uploaded to the IoT platform in the form of video or picture, so that the IoT platform can perform fixed analysis on the target section. In some embodiments, the uploading process is periodic, that is, the IoT platform can update the detected obstacle information in real time.

[0056] Figure 3 is a flowchart of a specific implementation method of step S23 in the embodiment of the present disclosure; in some embodiments, refer to Figure 3 , S23 includes:

[0057] S231, determining a distance between the obstacle and the audio output device according to the roadblock position of the obstacle and the current position of the audio output device;

[0058] S232: When the distance is less than or equal to a preset distance threshold, generate corresponding prompt information and send it to an audio output device.

[0059] In this embodiment, the IoT platform obtains obstacle information by analyzing and annotating the road section image, and the annotated content includes: the road width, length, and obstacle information of the target road section. Among them, the obstacle information includes: obstacles, the roadblock position of the obstacle, and whether the obstacle information will cause obstacles to the target user. In one example, the IoT platform analyzes and annotates the road section image of the target road section, and the annotated content includes: the electric pole on the first side of the target road section, and the bicycle parked on the roadside at the intersection. Further, when a new item appears in the target road section, the IoT platform analyzes in real time the coordinates of the item corresponding to the roadblock position in the target road section, and whether it will cause obstacles to the target user. When the target user approaches a new item that appears in the target road section, prompt information is generated based on this information and sent to the audio output device.

[0060] The IoT platform determines the distance between the obstacle and the audio output device by the difference between the obstacle position and the current position of the acquired audio output device. When the distance is less than or equal to the preset distance threshold, a corresponding prompt message is generated and sent to the audio output device. The prompt message is used for the audio output device to play the obstacle avoidance information voice. The obstacle avoidance information voice can accurately locate the obstacle through the IoT platform according to the needs of the target user.

[0061] In addition, it is worth noting that the embodiments of the present disclosure do not impose any special restrictions on the marking form of the roadblock position, which can be a coordinate marking or a relative reference position marking to a fixed object.

[0062] In some embodiments, the corresponding prompt information generated by the IoT platform can be obtained by converting the obstacle information according to a preset fixed template. In one example, the IoT platform analyzes and annotates the road section image to obtain obstacle information, and the obstacle information includes: a telephone pole, the roadblock position of the telephone pole is (x, y), and the telephone pole will cause an obstacle to the target user; a preset fixed template is: "there is an "obstacle" at the "roadblock position", and "whether" it will cause an obstacle to the target user. Therefore, the prompt information determined according to the preset fixed template and the obstacle information is: there is a telephone pole at the (x, y) position, which will cause an obstacle to the target user.

[0063] In some embodiments, the obstacle avoidance prompt method for the IoT platform further includes:

[0064] In response to the obstacle existing on the target road section, determining a target user carrying the audio output device from the road section image;

[0065] Determining an action state of the target user in the road segment image;

[0066] When the action state is a collision state, an alarm message is sent to the associated device.

[0067] In an embodiment of the present disclosure, the Internet of Things platform can associate at least one associated device with the audio output device of the target user. In some embodiments, the associated device can be a mobile phone, a computer, etc., and the interaction between the Internet of Things platform and the associated device can be achieved by associating family numbers, APPs (applications), and sending and receiving video images.

[0068] When the IoT platform determines that the action state of the target user in the road segment image is a collision state, the collision state can specifically represent the state of the target user falling or colliding with an obstacle, and the IoT platform sends an alarm message to the associated device, that is, the alarm message is linked with the associated device. In some embodiments, the IoT platform can show the associated device a video or picture of the real-time road segment image of the road segment where the target object currently reporting the obstacle is located, that is, the alarm message can be used to instruct the monitoring page of the associated device to play the road segment image of the target camera device.

[0069] In one example, the warning information is pushed by the IoT platform to the address registered for the family number, and the user corresponding to the family number can receive the road section image of the target camera device to determine whether assistance is needed for the target user.

[0070] Figure 4 A flowchart of an obstacle avoidance prompt method for an audio output device provided in an embodiment of the present disclosure. Figure 4 The present disclosure provides an obstacle avoidance prompting method for an audio output device, comprising:

[0071] S41, receiving prompt information sent by the IoT platform;

[0072] S42. Play corresponding obstacle avoidance information voice according to the prompt information; the obstacle avoidance information voice is used to instruct the target user to avoid obstacles.

[0073] In an embodiment of the present disclosure, a visually impaired target user carries an audio output device, and a camera device collects road section images of each road section in real time. When the IoT platform recognizes that the target user is moving within the road section image, the audio output device receives the prompt information sent by the IoT platform and plays the corresponding obstacle avoidance information voice to the target object, thereby providing accurate voice guidance to the target user to avoid dangerous objects.

[0074] In some embodiments, before S41, the method further includes:

[0075] Determine current location;

[0076] The current location is sent to the IoT platform.

[0077] In this embodiment, the audio output device has a positioning function, which can determine the current position of the target user carrying the audio output device in real time, and send the current position to the Internet of Things platform so that the Internet of Things platform can locate the target object in real time.

[0078] In some embodiments, the obstacle avoidance prompt method for an audio output device further includes:

[0079] In response to the target user's request for help, a communication request is sent to the associated device.

[0080] In the embodiment of the present disclosure, when the target user needs help, the help instruction sends a communication request to the associated device. The help instruction can be a voice instruction to trigger the family number connection function, and at the same time link the video monitoring page to push and play the real-time road section image of the target camera device, so that the user of the associated device can communicate with the target user through the voice system to determine whether help can be provided to the target user.

[0081] The embodiments of the first and second aspects of the present disclosure allow a visually impaired target user to carry an audio output device, and a camera device to collect road section images of each road section in real time. When the IoT platform recognizes that the target user is moving within the road section image, precise voice guidance can be provided to the target user to avoid dangerous objects.

[0082] Thirdly, refer to Figure 5 , an embodiment of the present disclosure provides an electronic device, comprising:

[0083] One or more processors 501;

[0084] A memory 502, on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above obstacle avoidance prompt methods;

[0085] One or more I / O interfaces 503 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0086] Among them, the processor 501 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 503 is connected between the processor 501 and the memory 502, and can realize information interaction between the processor 501 and the memory 502, including but not limited to a data bus (Bus), etc.

[0087] Fourthly, refer to Figure 6 The embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, any one of the above-mentioned obstacle avoidance prompt methods is implemented.

[0088] In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through specific embodiments:

[0089] Figure 7 A schematic diagram of the structure of an obstacle avoidance prompt system provided by an embodiment of the present disclosure. The obstacle avoidance prompt system is used to execute the obstacle avoidance prompt method, referring to Figure 7 The obstacle avoidance prompt system includes: a target user carrying an audio output device, an Internet of Things platform, and a target camera device.

[0090] The IoT platform includes: target camera device management module, voice conversion module, road section image analysis module and video processing module. The target camera device can be used for road section image capture and real-time road section image video shooting.

[0091] The target user carrying an audio output device moves to the target road section, and the audio output device reports the location (current location) to the IoT platform. The IoT platform searches for target camera devices near the current location through the target camera device management module, and requests to obtain the latest road section image of the target camera device; the road section image analysis module analyzes the road section image to obtain obstacle information; the voice conversion module obtains prompt information based on the road section image and obstacle information, and sends the prompt information to the audio output device; in addition, the IoT platform can also send an alarm message to the associated device through the video processing module when the target user's action state in the road section image is a collision state, and send the video or picture of the road section image to the associated device; by allowing the visually impaired target user to carry an audio output device, the camera device collects the road section images of each road section in real time. When the IoT platform recognizes that the target user is moving in the road section image, it can accurately guide the target user to avoid dangerous objects through voice guidance.

[0092] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0093] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. An obstacle avoidance prompt method, used in an IoT platform, comprising: Receiving a road section image of a target road section captured by a target camera device; Determining obstacle information of the target road section according to the road section image; The obstacle information of the target road section represents the situation of obstacles on the target road section; In response to the existence of the obstacle on the target road section, corresponding prompt information is generated and sent to the audio output device; the prompt information is used for the audio output device to play the obstacle avoidance information voice.

2. The obstacle avoidance prompting method according to claim 1, wherein: Before generating corresponding prompt information in response to the obstacle existing on the target road section and sending it to the audio output device, the method further includes: Get the current position of the audio output device; The road section where the current position is located is determined as the target road section, and the camera device that can capture an image of the target road section is determined as the target camera device.

3. The obstacle avoidance prompting method according to claim 2, wherein: Receiving a road section image of a target road section collected by a camera device, including: When the current position is continuously located in the target road section, the road section image of the target road section captured by the target camera device is periodically received.

4. The obstacle avoidance prompting method according to claim 1, wherein: In response to the obstacle existing on the target road section, generating corresponding prompt information and sending it to an audio output device, including: Determining a distance between the obstacle and the audio output device according to a roadblock position of the obstacle and a current position of the audio output device; When the distance is less than or equal to the preset distance threshold, a corresponding prompt message is generated and sent to the audio output device.

5. The obstacle avoidance prompting method according to claim 1, wherein: Also includes: In response to the obstacle existing on the target road section, determining a target user carrying the audio output device from the road section image; Determining an action state of the target user in the road segment image; When the action state is a collision state, an alarm message is sent to the associated device.

6. An obstacle avoidance prompt method for an audio output device, comprising: Receive prompt information sent by the IoT platform; According to the prompt information, play the corresponding obstacle avoidance information voice; The obstacle avoidance information voice is used to instruct the target user to avoid obstacles.

7. The obstacle avoidance prompting method according to claim 6, wherein: Before receiving the prompt information sent by the IoT platform, it also includes: Determine current location; The current location is sent to the IoT platform.

8. The obstacle avoidance prompting method according to claim 6, wherein: Also includes: In response to the target user's request for help, a communication request is sent to the associated device.

9. An electronic device, comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle avoidance prompt method according to any one of claims 1 to 8.

10. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the obstacle avoidance prompt method according to any one of claims 1 to 8 is implemented.

Images