Environment safety awareness method and related device

Abstract

This application discloses an environmental safety perception method and related equipment. It can acquire environmental images surrounding the current location of a target object, the environmental images including the target object; extract human eye images corresponding to the target object from the environmental images; perform frontal scene recognition on the human eye images to obtain frontal scene recognition results, the frontal scene being the scene within the target object's current field of vision; based on the frontal scene recognition results, determine whether there are potential hazards surrounding the target object's current location; if potential hazards exist, send safety warning information to the target object. This application can analyze the environmental image received by the human eye from the human eye image, thereby obtaining frontal scene recognition results, and alerting the target object to potential risks ahead or to the side based on the frontal scene recognition results, thus helping users avoid potential dangers in the surrounding environment.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an environmental security perception method and related equipment. Background Technology

[0002] With the development of internet technology and the rapid popularization of mobile devices, mobile devices have become indispensable tools for production, entertainment, and communication in people's daily lives. People are accustomed to obtaining various information from smartphones, and many have become "phubbers." When people's eyes and attention are focused on smartphones, they easily overlook potential dangers in their surroundings.

[0003] In current technologies, the rear camera of a smartphone can be used to capture environmental images and display them on the phone screen, giving users the ability to "see through" the smartphone. However, the field of view of the scene captured in this way is relatively limited, because during normal walking, the user can generally only capture the scene on the ground, and therefore cannot provide warnings about potential dangers in front and to the sides. Summary of the Invention

[0004] This application provides an environmental safety sensing method and related equipment. The related equipment may include an environmental safety sensing device, electronic equipment, computer-readable storage medium, and computer program products, which can help users avoid potential dangers in the surrounding environment.

[0005] This application provides an environmental safety perception method, including:

[0006] Collect an environmental image of the area surrounding the current location of the target object, the environmental image including the target object;

[0007] Extract the human eye image corresponding to the target object from the environmental image;

[0008] Perform frontal scene recognition on the human eye image to obtain frontal scene recognition results, wherein the frontal scene is the scene within the current field of vision of the target object;

[0009] Based on the front scene recognition results, it is determined whether there are potential danger factors around the current location of the target object;

[0010] If there are potential hazards in the vicinity of the target object's current location, a safety alert message is sent to the target object.

[0011] Accordingly, embodiments of this application provide an environmental safety sensing device, including:

[0012] The acquisition unit is used to acquire environmental images of the surrounding area of ​​the current location of the target object, wherein the environmental images include the target object;

[0013] The extraction unit is used to extract the human eye image corresponding to the target object from the environmental image;

[0014] The recognition unit is used to perform front scene recognition on the human eye image to obtain a front scene recognition result, wherein the front scene is the scene within the current field of vision of the target object;

[0015] The determining unit is used to determine, based on the front scene recognition results, whether there are potential danger factors around the current location of the target object;

[0016] The sending unit is used to send a safety warning message to the target object if there are potential hazards in the vicinity of the target object's current location.

[0017] Optionally, in some embodiments of this application, the environmental safety sensing device may further include a segmentation unit and a scene recognition unit, as follows:

[0018] The segmentation unit is used to perform face segmentation processing on the environmental image in order to extract the rear environmental image from the environmental image;

[0019] A scene recognition unit is used to perform rear scene recognition on the rear environment image to obtain a rear scene recognition result, wherein the rear scene is a scene outside the current field of view of the target object;

[0020] The determining unit can be used to determine whether there are potential danger factors around the current location of the target object based on the front scene recognition result and the rear scene recognition result.

[0021] Optionally, in some embodiments of this application, the determining unit may include a risk determination subunit and a first determining subunit, as follows:

[0022] The risk assessment subunit is used to assess the risk of the front scene recognition result and the rear scene recognition result according to a preset hazard assessment strategy.

[0023] The first determining subunit is used to determine, based on the risk assessment results, whether there are any potential dangers around the current location of the target object.

[0024] Optionally, in some embodiments of this application, the identification unit may include an extraction subunit and an identification subunit, as follows:

[0025] The extraction subunit is used to extract from the human eye image the environmental image received by the target object for the front scene;

[0026] The identification subunit is used to perform object identification on the environmental image to obtain the target objects present in the front scene.

[0027] Optionally, in some embodiments of this application, the determining unit may include a calculation subunit and a second determining subunit, as follows:

[0028] The computing subunit is used to calculate the relative position between the target object and the target subject based on the environmental image;

[0029] The second determining subunit is used to determine whether there are potential danger factors around the current location of the target object based on the relative position and the object type corresponding to the target object.

[0030] Optionally, in some embodiments of this application, the extraction subunit may specifically be used to perform edge detection on the human eye image to obtain a human eye corneal region image corresponding to the target object; and to obtain an environmental image of the target object receiving the front scene based on the human eye corneal region image.

[0031] An electronic device provided in this application includes a processor and a memory. The memory stores multiple instructions, and the processor loads the instructions to execute the steps in the environmental safety perception method provided in this application.

[0032] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps in the environmental security perception method provided in this application.

[0033] Furthermore, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in the environmental security awareness method provided in embodiments of this application.

[0034] This application provides an environmental safety perception method and related equipment. It can acquire environmental images surrounding the current location of a target object, including the target object; extract human eye images corresponding to the target object from the environmental images; perform frontal scene recognition on the human eye images to obtain frontal scene recognition results, where the frontal scene is the scene within the target object's current field of vision; determine whether there are potential hazards surrounding the target object's current location based on the frontal scene recognition results; and if potential hazards exist, send safety alert information to the target object. This application can analyze the environmental image received by the human eye from the human eye image, thereby obtaining frontal scene recognition results, and alerting the target object to potential risks ahead or to the side based on the frontal scene recognition results, thus helping users avoid potential dangers in the surrounding environment. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1a This is a schematic diagram of a scenario for the environmental safety perception method provided in an embodiment of this application;

[0037] Figure 1b This is a flowchart of the environmental safety perception method provided in the embodiments of this application;

[0038] Figure 1c This is an illustrative diagram of the environmental safety perception method provided in the embodiments of this application;

[0039] Figure 1d This is another flowchart of the environmental safety perception method provided in the embodiments of this application;

[0040] Figure 2 This is another flowchart of the environmental safety perception method provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the structure of the environmental safety sensing device provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] This application provides an environmental safety sensing method and related equipment. The related equipment may include an environmental safety sensing device, an electronic device, a computer-readable storage medium, and a computer program product. Specifically, the environmental safety sensing device may be integrated into an electronic device, which may be a terminal or a server, etc.

[0045] like Figure 1a As shown, the environmental security perception method is implemented jointly by a terminal and a server as an example. The environmental security perception system provided in this application includes a terminal 10 and a server 11, etc.; the terminal 10 and the server 11 are connected through a network, such as through a wired or wireless network, etc., wherein the environmental security perception device can be integrated into the terminal.

[0046] Terminal 10 can be used to: acquire environmental images of the surrounding area of ​​the target object's current location, the environmental images including the target object; extract human eye images corresponding to the target object from the environmental images; perform front-side scene recognition on the human eye images to obtain front-side scene recognition results, the front-side scene being the scene within the target object's current field of vision; determine whether there are potential hazards around the target object's current location based on the front-side scene recognition results; and send safety alert information to the target object if potential hazards exist around the target object's current location. Terminal 10 may include mobile phones, tablets, smartwatches, smart bracelets, etc. A client may also be installed on terminal 10, which may be an application client, etc.

[0047] Server 11 can be used to: receive an environmental image sent by terminal 10; extract the human eye image corresponding to the target object from the environmental image; perform front-side scene recognition on the human eye image to obtain a front-side scene recognition result, wherein the front-side scene is the scene within the current field of vision of the target object; and send the front-side scene recognition result to terminal 10. Server 11 can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0048] The environmental safety perception method provided in this application relates to computer vision technology in the field of artificial intelligence.

[0049] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, machine learning / deep learning, autonomous driving, and intelligent transportation.

[0050] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0051] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0052] This embodiment will be described from the perspective of an environmental safety sensing device, which can be integrated into an electronic device, such as a server or a terminal.

[0053] like Figure 1b As shown, the specific process of this environmental safety perception method can be as follows:

[0054] 101. Collect an environmental image of the area surrounding the current location of the target object, wherein the environmental image includes the target object.

[0055] The target object can be a pedestrian on the road or the pedestrian's face; this embodiment does not impose specific limitations. The environmental image can be an image captured by the front-facing camera of a smart mobile terminal held by the target object. The front-facing camera can be camera hardware integrated on the smart mobile terminal, located on the same side as the main display screen of the smart mobile terminal.

[0056] In a specific scenario, the front-facing camera of a smart mobile terminal (such as a smartphone) can be used to collect environmental images of the surrounding environment where the target object is currently located. These environmental images include the target object's face. Therefore, the reflected image of the human eyeball in front of the scene can be extracted from the environmental image, and the reflected image can be restored to obtain the environmental image received by the target object. Based on the environmental image, it can be analyzed whether there are potential dangers in the environment where the target object is currently located.

[0057] The environmental safety perception method provided in this application enables smartphones to perceive their surroundings without altering current smartphone hardware. Specifically, the human eye has a relatively large field of view, receiving a wide range of visual information. However, the human brain can only process a portion of the objects within its field of view, especially when attention is focused on a phone screen, making it difficult to perceive potential risks in the environment. This application addresses this issue by utilizing the front-facing camera found on most smartphones. Using the front-facing camera as the hardware foundation, it acquires environmental reflections from the surface of the human eye to perceive objects outside the front-facing camera's field of view. This provides a wider field of view and allows for rapid deployment of environmental perception functionality on most existing smartphone hardware, supplementing human environmental perception capabilities in specific scenarios and helping users avoid potential dangers.

[0058] 102. Extract the human eye image corresponding to the target object from the environmental image.

[0059] Optionally, in this embodiment, the step of "extracting the human eye image corresponding to the target object from the environmental image" may include:

[0060] The environmental image is subjected to face segmentation processing to extract the rear environmental image and the face image from the environmental image;

[0061] Extract the human eye image corresponding to the target object from the face image.

[0062] In some embodiments, the face segmentation processing of environmental images can specifically be the face recognition processing of environmental images. Specifically, face recognition processing of environmental images can be performed using a neural network model, such as a Visual Geometry Group Network (VGGNet), a Residual Network (ResNet), and a Dense Convolutional Network (DenseNet), etc. However, it should be understood that the neural network in this embodiment is not limited to the types listed above.

[0063] In some other embodiments, the face segmentation processing of the environmental image may specifically involve extracting the face image from the environmental image and determining the remaining image area as the rear environmental image.

[0064] It is understood that in some embodiments, a portion of the image region corresponding to the target object can be extracted from the environmental image, and the remaining image region can be determined as the rear environmental image.

[0065] After obtaining the face image, the face image can be further identified and segmented into the eye region, and the eye region can be identified as the eye image corresponding to the target object.

[0066] 103. Perform front-side scene recognition on the human eye image to obtain a front-side scene recognition result, wherein the front-side scene is the scene within the current field of vision of the target object.

[0067] Optionally, in this embodiment, the step "performing frontal scene recognition on the human eye image to obtain frontal scene recognition results" may include:

[0068] The target object receives an environmental image of the front scene from the human eye image;

[0069] Object recognition is performed on the environmental image to obtain the target objects present in the front scene.

[0070] Specifically, object recognition in environmental images can involve identifying objects present in the images, such as utility poles or cars. The identified target objects can be utility poles on the road, cars, or objects falling from heights; this embodiment does not impose any limitations on this.

[0071] Optionally, in this embodiment, the step "extracting the environmental image of the target object received from the human eye image for the front scene" may include:

[0072] Edge detection is performed on the human eye image to obtain the corneal region image of the target object;

[0073] Based on the image of the corneal region of the human eye, the environmental image of the target object received for the front scene is obtained.

[0074] Before performing edge detection on the human eye image, image enhancement processing can be performed to improve its resolution. Image data enhancement can include histogram equalization, sharpening, smoothing, and multi-frame merging. Specifically, multi-frame merging can involve extracting the corresponding human eye image from multiple acquired environmental images and then merging these multiple human eye images.

[0075] In addition, before performing edge detection on the human eye image, image distortion correction processing can be performed on the human eye image to obtain a distortion-free human eye image.

[0076] In this embodiment, edge detection of the human eye image can be performed using contour detection algorithms and Hough transform to determine the corneal region within the eyeball, extract the corneal region image, and receive the corneal region image as the target object for the environmental image of the foreground scene. The Hough transform is a commonly used edge calculation method in digital image processing.

[0077] 104. Based on the front scene recognition results, determine whether there are potential danger factors around the current location of the target object.

[0078] Optionally, in this embodiment, the step "based on the front scene recognition result, determine whether there are potential danger factors around the current location of the target object" may include:

[0079] Based on the environmental image, calculate the relative position between the target object and the target subject;

[0080] Based on the relative position and the object type corresponding to the target object, determine whether there are potential hazards around the current location of the target object.

[0081] The distance between the target object and the target subject can be calculated based on the environmental imagery, thereby determining their relative positions.

[0082] The object type of the target object can include objects that have potential hazards and objects that do not have potential hazards.

[0083] In one specific embodiment, if the distance between the target object and the target object is less than a preset safe distance, and the target object is a pre-defined potentially hazardous object, then it can be determined that there are potential hazards around the current location of the target object. The preset safe distance can be set according to actual conditions, for example, it can be set to 1 meter. Pre-defined potentially hazardous objects can be cars, utility poles, etc.

[0084] In addition, this embodiment can also use the accelerometer of the smart mobile terminal to determine the current motion state of the target object, and combine the object type, motion state, and relative position of the target object to determine whether there are potential dangers around the current location of the target object.

[0085] Optionally, in this embodiment, the environmental safety perception method may further include:

[0086] The environmental image is subjected to face segmentation processing to extract the rear environmental image from the environmental image;

[0087] The rear environment image is subjected to rear scene recognition to obtain a rear scene recognition result, wherein the rear scene is the scene outside the current field of view of the target object;

[0088] The step "Based on the front scene recognition results, determine whether there are potential hazards around the current location of the target object" may include:

[0089] Based on the front scene recognition results and the rear scene recognition results, it is determined whether there are potential danger factors around the current location of the target object.

[0090] The step "perform rear scene recognition on the rear environment image to obtain rear scene recognition result" can specifically be to identify objects in the rear environment image, such as cars. In this embodiment, objects identified from the rear environment image can also be identified as target objects.

[0091] Specifically, rear scene recognition can be performed on the rear environment image using a scene recognition model, which can be a neural network model.

[0092] Optionally, in this embodiment, the step "determining whether there are potential hazards around the current location of the target object based on the front scene recognition result and the rear scene recognition result" may include:

[0093] Based on a preset hazard assessment strategy, risk assessment is performed on the front scene recognition results and the rear scene recognition results;

[0094] Based on the risk assessment results, determine whether there are any potential hazards in the vicinity of the target object's current location.

[0095] The preset hazard assessment strategy can be set according to the actual situation. For example, if the utility pole is 1 meter in front of the target object, then there are potential hazards around the target object's current location, and an early warning needs to be issued to the target object.

[0096] 105. If there are potential hazards in the vicinity of the target object's current location, send a safety warning message to the target object.

[0097] There are various ways to provide safety alerts to the target object. For example, the safety alert can be displayed on the screen of the smart mobile terminal held by the target object, or it can be reminded to the target object through a speaker. This embodiment does not limit this.

[0098] This application provides a "line-of-sight environment perception system for handheld mobile terminals." By segmenting the environmental image captured by the target object's handheld smart mobile terminal, it obtains the image of the human eye and the image of the environment behind it. Utilizing the virtual image of environmental reflection on the surface of the human eye, it analyzes and obtains the reflected image from the eyeball image, reconstructing the environmental image received by the human eyeball. This allows it to obtain environmental information in front of the target object (which can be considered a front-side scene cue). Furthermore, by combining this with other real images of the scene in the rear-side environment image, it obtains environmental information outside the target object's current field of vision (which can be considered a rear-side scene cue). Based on the front-side and rear-side scene cuees, it can perceive potential risks in the surrounding environment of the target object's current location, thus providing the user of the smart mobile terminal with an environmental briefing to help them avoid potential risks. Specifically, line-of-sight environment perception refers to the ability to perceive real images of scenes outside the camera's field of view.

[0099] This application can alert users to potential risks ahead or to the side while they are walking and looking at their phone screen, such as oncoming vehicles, stationary utility poles, potholes in the ground ahead, falling objects overhead, traffic lights on pedestrian crossings, etc., and analyze the results to determine whether to display them on the phone screen. This allows the application to provide alerts and warnings to users before they even perceive the objects themselves.

[0100] Specifically, this application imposes certain hardware requirements on the smart mobile terminal: the mobile terminal is required to have a front-facing camera, such as... Figure 1c As shown, this application can combine the field of view of the front-facing camera of the smart mobile terminal with the field of view of the human eye corresponding to the acquired image of the human eye reflection to perceive the surrounding environment of the target object. The application scenario of this application can be seen in the use of a smart mobile terminal while walking.

[0101] In one specific embodiment, such as Figure 1d The diagram illustrates the process by which the environmental safety perception method provided in this application provides safety prompts to users. The specific details are as follows:

[0102] (1) The environmental images are collected in real time by the front camera of the smart mobile terminal held by the target object, and the environmental images are transmitted to the processor for face recognition. After processing, the environmental images are divided into face images and rear environmental images.

[0103] (2) Further use object detection algorithm and scene detection algorithm to identify objects in the rear environment image obtained in step (1) and mark them as rear scene prompts;

[0104] (3) Further identify and segment the face image obtained in step (1) into eye images, and further perform anti-distortion correction and multi-frame synthesis on the eye images to finally obtain clear and distortion-free eye images.

[0105] (4) For the human eye image obtained in step (3), further determine the corneal region in the eyeball by contour detection combined with Hough transform, and extract the human eye corneal region image;

[0106] (5) Perform object recognition and scene recognition on the corneal region image of the human eye obtained in step (4) to finally obtain the objects and scenes within the field of vision of the human eye and mark them as front scene prompts;

[0107] (6) In steps (2) and (5), the rear scene prompts within the field of view of the front camera and the front scene prompts within the field of view of the human eye are obtained. After the preset danger judgment strategy is used to make a decision, if there are potential danger factors (such as oncoming cars, utility poles on the road, falling objects from high altitudes, etc.), the judgment result will be displayed on the mobile terminal output device. The output device includes, but is not limited to, screens, speakers, and wireless connection devices.

[0108] Existing environmental perception technologies mostly rely on additional sensors or simply use images captured by cameras to provide user alerts without additional processing. This application, building upon existing mobile smart terminals, utilizes the front-facing camera hardware found in most devices to acquire reflected images from the surface of the human eye and other environmental images obtained by the front-facing camera. After a series of processing steps, a briefing of the current environment is generated, and potential hazards are directly fed back to the user. Compared to existing methods, this approach offers advantages such as lower application and deployment costs and a wider range of environmental perception capabilities.

[0109] This application can, to some extent, reduce the dangers of traffic accidents, collisions with walls, and missteps caused by "smartphone addicts" on the road due to their lack of concentration while using smartphones.

[0110] As can be seen from the above, this embodiment can collect environmental images around the current location of a target object, the environmental images including the target object; extract the human eye image corresponding to the target object from the environmental images; perform frontal scene recognition on the human eye image to obtain a frontal scene recognition result, the frontal scene being the scene within the current field of vision of the target object; based on the frontal scene recognition result, determine whether there are potential hazards around the current location of the target object; if there are potential hazards around the current location of the target object, send a safety warning message to the target object. This application can analyze the environmental image received by the human eye from the human eye image, and then obtain the frontal scene recognition result, and warn the target object of potential risks in front or to the side based on the frontal scene recognition result, thus helping users avoid potential dangers in the surrounding area.

[0111] Based on the method described in the preceding embodiments, the following will provide a more detailed explanation by taking the specific integration of the environmental safety sensing device into a terminal as an example.

[0112] This application provides an environmental safety perception method, such as... Figure 2 As shown, the specific process of this environmental safety perception method can be as follows:

[0113] 201. The terminal collects environmental images of the surrounding area of ​​the current location of the target object, and the environmental images include the target object.

[0114] The target object can be a pedestrian on the road or the pedestrian's face; this embodiment does not impose specific limitations on this. The environmental image can be an image captured by the front-facing camera of a smart mobile terminal held by the target object.

[0115] 202. The terminal performs face segmentation processing on the environmental image to extract the rear environmental image and the face image from the environmental image, and extracts the eye image corresponding to the target object from the face image.

[0116] In some embodiments, the face segmentation processing of the environmental image can specifically be face recognition processing of the environmental image. In other embodiments, the face segmentation processing of the environmental image can specifically be extracting the face image from the environmental image and determining the remaining image area as the rear environmental image.

[0117] It is understood that in some embodiments, a portion of the image region corresponding to the target object can be extracted from the environmental image, and the remaining image region can be determined as the rear environmental image.

[0118] After obtaining the face image, the face image can be further identified and segmented into the eye region, and the eye region can be identified as the eye image corresponding to the target object.

[0119] 203. The terminal performs rear scene recognition on the rear environment image to obtain a rear scene recognition result, wherein the rear scene is the scene outside the current field of view of the target object.

[0120] The step "perform rear scene recognition on the rear environment image to obtain rear scene recognition result" can specifically be to identify objects in the rear environment image, such as cars. In this embodiment, objects identified from the rear environment image can also be identified as target objects.

[0121] 204. The terminal performs frontal scene recognition on the human eye image to obtain a frontal scene recognition result, wherein the frontal scene is the scene within the current field of vision of the target object.

[0122] Optionally, in this embodiment, the step "performing frontal scene recognition on the human eye image to obtain frontal scene recognition results" may include:

[0123] The target object receives an environmental image of the front scene from the human eye image;

[0124] Object recognition is performed on the environmental image to obtain the target objects present in the front scene.

[0125] Specifically, object recognition in environmental images can involve identifying objects present in the images, such as utility poles or cars. The identified target objects can be utility poles on the road, cars, or objects falling from heights; this embodiment does not impose any limitations on this.

[0126] Optionally, in this embodiment, the step "extracting the environmental image of the target object received from the human eye image for the front scene" may include:

[0127] Edge detection is performed on the human eye image to obtain the corneal region image of the target object;

[0128] Based on the image of the corneal region of the human eye, the environmental image of the target object received for the front scene is obtained.

[0129] 205. Based on the front scene recognition results and the rear scene recognition results, the terminal determines whether there are potential danger factors around the current location of the target object.

[0130] Optionally, in this embodiment, the step "determining whether there are potential hazards around the current location of the target object based on the front scene recognition result and the rear scene recognition result" may include:

[0131] Calculate the relative position between the target object and the target host;

[0132] Based on the relative position and the object type corresponding to the target object, determine whether there are potential hazards around the current location of the target object.

[0133] In one specific embodiment, if the distance between the target object and the target object is less than a preset safe distance, and the target object is a pre-defined potentially hazardous object, then it can be determined that there are potential hazards around the current location of the target object. The preset safe distance can be set according to actual conditions, for example, it can be set to 1 meter. Pre-defined potentially hazardous objects can be cars, utility poles, etc.

[0134] In addition, this embodiment can also use the accelerometer of the smart mobile terminal to determine the current motion state of the target object, and combine the object type, motion state, and relative position of the target object to determine whether there are potential dangers around the current location of the target object.

[0135] Optionally, in this embodiment, the step "determining whether there are potential hazards around the current location of the target object based on the front scene recognition result and the rear scene recognition result" may include:

[0136] Based on a preset hazard assessment strategy, risk assessment is performed on the front scene recognition results and the rear scene recognition results;

[0137] Based on the risk assessment results, determine whether there are any potential hazards in the vicinity of the target object's current location.

[0138] The preset hazard assessment strategy can be set according to the actual situation. For example, if the utility pole is 1 meter in front of the target object, then there are potential hazards around the target object's current location, and an early warning needs to be issued to the target object.

[0139] 206. If there are potential hazards in the vicinity of the target object's current location, the terminal sends a safety alert message to the target object.

[0140] There are various ways to provide safety alerts to the target object. For example, the safety alert can be displayed on the screen of the smart mobile terminal held by the target object, or it can be reminded to the target object through a speaker. This embodiment does not limit this.

[0141] As can be seen from the above, this embodiment can collect environmental images around the current location of a target object via a terminal. The environmental images include the target object. Face segmentation processing is performed on the environmental images to extract rear environmental images and face images, and the eye image corresponding to the target object is extracted from the face image. Rear scene recognition is performed on the rear environmental images to obtain rear scene recognition results, where the rear scene is the scene outside the target object's current field of vision. Front scene recognition is performed on the eye images to obtain front scene recognition results, where the front scene is the scene within the target object's current field of vision. Based on the front scene recognition results and the rear scene recognition results, it is determined whether there are potential hazards around the current location of the target object. If potential hazards exist around the current location of the target object, the terminal sends a safety warning message to the target object. This application can analyze the environmental image received by the human eye from the eye image, and then obtain the front scene recognition result. Based on the front scene recognition result, it warns the target object of potential risks in front or to the side, thus helping users avoid potential dangers in the surrounding area.

[0142] To better implement the above methods, embodiments of this application also provide an environmental safety sensing device, such as... Figure 3 As shown, the environmental safety sensing device may include a data acquisition unit 301, an extraction unit 302, an identification unit 303, a determination unit 304, and a transmission unit 305, as follows:

[0143] (1) Acquisition unit 301;

[0144] The acquisition unit is used to acquire environmental images of the surrounding area of ​​the current location of the target object, and the environmental images include the target object.

[0145] (2) Extraction unit 302;

[0146] An extraction unit is used to extract the human eye image corresponding to the target object from the environmental image.

[0147] (3) Identification unit 303;

[0148] The recognition unit is used to perform frontal scene recognition on the human eye image to obtain a frontal scene recognition result, wherein the frontal scene is the scene within the current field of vision of the target object.

[0149] Optionally, in some embodiments of this application, the identification unit may include an extraction subunit and an identification subunit, as follows:

[0150] The extraction subunit is used to extract from the human eye image the environmental image received by the target object for the front scene;

[0151] The identification subunit is used to perform object identification on the environmental image to obtain the target objects present in the front scene.

[0152] (4) Determine unit 304;

[0153] The determining unit is used to determine, based on the front scene recognition results, whether there are potential danger factors around the current location of the target object.

[0154] Optionally, in some embodiments of this application, the determining unit may include a calculation subunit and a second determining subunit, as follows:

[0155] The computing subunit is used to calculate the relative position between the target object and the target subject based on the environmental image;

[0156] The second determining subunit is used to determine whether there are potential danger factors around the current location of the target object based on the relative position and the object type corresponding to the target object.

[0157] Optionally, in some embodiments of this application, the extraction subunit may specifically be used to perform edge detection on the human eye image to obtain a human eye corneal region image corresponding to the target object; and to obtain an environmental image of the target object receiving the front scene based on the human eye corneal region image.

[0158] Optionally, in some embodiments of this application, the environmental safety sensing device may further include a segmentation unit and a scene recognition unit, as follows:

[0159] The segmentation unit is used to perform face segmentation processing on the environmental image in order to extract the rear environmental image from the environmental image;

[0160] A scene recognition unit is used to perform rear scene recognition on the rear environment image to obtain a rear scene recognition result, wherein the rear scene is a scene outside the current field of view of the target object;

[0161] The determining unit can be used to determine whether there are potential danger factors around the current location of the target object based on the front scene recognition result and the rear scene recognition result.

[0162] Optionally, in some embodiments of this application, the determining unit may include a risk determination subunit and a first determining subunit, as follows:

[0163] The risk assessment subunit is used to assess the risk of the front scene recognition result and the rear scene recognition result according to a preset hazard assessment strategy.

[0164] The first determining subunit is used to determine, based on the risk assessment results, whether there are any potential dangers around the current location of the target object.

[0165] (5) Transmitting unit 305;

[0166] The sending unit is used to send a safety warning message to the target object if there are potential hazards in the vicinity of the target object's current location.

[0167] As can be seen from the above, this embodiment can acquire environmental images around the current location of the target object through the acquisition unit 301, the environmental images including the target object; extract human eye images corresponding to the target object from the environmental images through the extraction unit 302; perform front scene recognition on the human eye images through the recognition unit 303 to obtain front scene recognition results, the front scene being the scene within the current field of vision of the target object; determine whether there are potential dangers around the current location of the target object based on the front scene recognition results through the determination unit 304; if there are potential dangers around the current location of the target object, the sending unit 305 sends safety warning information to the target object. This application can analyze the environmental images received by the human eye from the human eye images, and then obtain the front scene recognition results, and warn the target object of potential risks in front or to the side based on the front scene recognition results, thus helping users avoid potential dangers in the surrounding area.

[0168] This application also provides an electronic device, such as... Figure 4 The diagram shows a structural schematic of an electronic device involved in an embodiment of this application. This electronic device can be a terminal or a server, specifically:

[0169] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0170] The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0171] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0172] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0173] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0174] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0175] The system acquires an environmental image of the area surrounding the target object's current location, the environmental image including the target object; extracts an eye image corresponding to the target object from the environmental image; performs frontal scene recognition on the eye image to obtain a frontal scene recognition result, the frontal scene being the scene within the target object's current field of vision; based on the frontal scene recognition result, determines whether there are potential hazards around the target object's current location; if there are potential hazards around the target object's current location, sends a safety warning message to the target object.

[0176] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0177] As can be seen from the above, this embodiment can collect environmental images around the current location of a target object, the environmental images including the target object; extract the human eye image corresponding to the target object from the environmental images; perform frontal scene recognition on the human eye image to obtain a frontal scene recognition result, the frontal scene being the scene within the current field of vision of the target object; based on the frontal scene recognition result, determine whether there are potential hazards around the current location of the target object; if there are potential hazards around the current location of the target object, send a safety warning message to the target object. This application can analyze the environmental image received by the human eye from the human eye image, and then obtain the frontal scene recognition result, and warn the target object of potential risks in front or to the side based on the frontal scene recognition result, thus helping users avoid potential dangers in the surrounding area.

[0178] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0179] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the environmental security awareness methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0180] The system acquires an environmental image of the area surrounding the target object's current location, the environmental image including the target object; extracts an eye image corresponding to the target object from the environmental image; performs frontal scene recognition on the eye image to obtain a frontal scene recognition result, the frontal scene being the scene within the target object's current field of vision; based on the frontal scene recognition result, determines whether there are potential hazards around the target object's current location; if there are potential hazards around the target object's current location, sends a safety warning message to the target object.

[0181] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0182] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0183] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the environmental safety perception methods provided in the embodiments of this application, the beneficial effects that any of the environmental safety perception methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0184] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations of the above-described environmental security awareness aspect.

[0185] The above provides a detailed description of an environmental safety sensing method and related equipment provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An environmentally safe perception method, characterized by, The method comprises the following steps: collecting an environment image around a current position of a target object, the environment image comprising the target object; extracting a human eye image corresponding to the target object from the environment image; performing front scene recognition on the human eye image to obtain a front scene recognition result, the front scene being a scene within a current field of view of the target object; determining whether there is a potential dangerous factor around the current position of the target object based on the front scene recognition result; if there is a potential dangerous factor around the current position of the target object, sending a safety prompt information to the target object; wherein the method further comprises: performing face segmentation processing on the environment image to extract a back environment image from the environment image; performing back scene recognition on the back environment image to obtain a back scene recognition result, the back scene being a scene outside the current field of view of the target object; the determining whether there is a potential dangerous factor around the current position of the target object based on the front scene recognition result comprises: determining whether there is a potential dangerous factor around the current position of the target object based on the front scene recognition result and the back scene recognition result.

2. The method of claim 1, wherein, the determining whether there is a potential dangerous factor around the current position of the target object based on the front scene recognition result and the back scene recognition result comprises: performing risk judgment on the front scene recognition result and the back scene recognition result according to a preset dangerous judgment strategy; determining whether there is a potential dangerous factor around the current position of the target object according to a risk judgment result.

3. The method of claim 1, wherein, the performing front scene recognition on the human eye image to obtain a front scene recognition result comprises: extracting an environment image received by the target object for the front scene from the human eye image; performing object recognition on the environment image to obtain a target object existing in the front scene.

4. The method of claim 3, wherein, the determining whether there is a potential dangerous factor around the current position of the target object based on the front scene recognition result comprises: calculating a relative position between the target object and the target object according to the environment image; determining whether there is a potential dangerous factor around the current position of the target object according to the relative position and an object type corresponding to the target object.

5. The method of claim 3, wherein, the extracting an environment image received by the target object for the front scene from the human eye image comprises: performing edge detection on the human eye image to obtain a human eye cornea region image corresponding to the target object; obtaining the environment image received by the target object for the front scene according to the human eye cornea region image.

6. An environmentally safe sensing device, characterized by, The method comprises the following steps: a collecting unit configured to collect an environment image around a current position of a target object, the environment image comprising the target object; an extracting unit configured to extract a human eye image corresponding to the target object from the environment image; an identifying unit configured to perform front scene recognition on the human eye image to obtain a front scene recognition result, the front scene being a scene within a current field of view of the target object; The determining unit is configured to determine whether there is a potential dangerous factor around the current position of the target object based on the front-side scene recognition result. The sending unit is configured to send safety prompt information to the target object if there is a potential dangerous factor around the current position of the target object. The dividing unit is configured to perform face segmentation processing on the environment image to extract a rear-side environment image from the environment image. The scene recognition unit is configured to perform rear-side scene recognition on the rear-side environment image to obtain a rear-side scene recognition result, the rear-side scene being a scene outside the current field of view of the target object. The determining unit is specifically configured to determine whether there is a potential dangerous factor around the current position of the target object based on the front-side scene recognition result and the rear-side scene recognition result.

7. An electronic device, comprising: The memory stores an application program, and the processor is configured to run the application program in the memory to perform operations in the environment safety perception method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, which are adapted to be loaded by the processor to perform steps in the environment safety perception method according to any one of claims 1 to 5.

9. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by the processor, implement steps in the environment safety perception method according to any one of claims 1 to 5.

Images