Safety early warning method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of intelligent headphones, and relates to a safety early warning method, which comprises the following steps: receiving road surface acquisition information acquired by image acquisition equipment; performing image preprocessing on the road surface acquisition information to obtain a preprocessed image; performing zebra crossing identification processing on the preprocessed image to obtain an image zebra crossing identification result; if the zebra crossing image does not exist in the image zebra crossing recognition result, ending the early warning operation; if the zebra crossing image exists in the image zebra crossing recognition result, obtaining current positioning information, and determining whether a map zebra crossing corresponding to the current positioning information exists in an electronic map; if no map zebra crossing exists in the electronic map, ending the early warning operation; and if the map zebra crossing exists in the electronic map, generating safety early warning information, and outputting the safety early warning information through the headset. According to the invention, the situation that pedestrians run the red light when playing mobile phones by lowering heads during traveling can be effectively avoided, and the traveling risk is greatly reduced.

Description

Technical Field

[0001] This application relates to the technical field of intelligent head-mounted earphones, and particularly to a safety warning method, device, computer device, and storage medium. Background Art

[0002] When a pedestrian uses a head-mounted earphone to look at a mobile phone and walks with the head down, the average visual field obtained by the eyes is only 5% of that in normal walking, and the average walking speed is also reduced by 16% to 33%. It is difficult to perceive the surrounding environment, which easily leads to traffic accidents.

[0003] When the existing head-mounted earphones on the market are in use, they do not have the function of safety reminder according to the surrounding environment. It is difficult for users to perceive the surrounding environment up, down, left, and right through hearing. Especially when looking at a mobile phone and listening to music while walking across the road, there are relatively large travel traffic safety risks. Summary of the Invention

[0004] The purpose of the embodiments of this application is to propose a safety warning method, device, computer device, and storage medium to solve the technical problem that it is difficult to focus on perceiving the surrounding environment when using a head-mounted earphone, resulting in walking safety problems.

[0005] To solve the above technical problem, the embodiments of this application provide a safety warning method, which adopts the following technical solutions:

[0006] Receive the road surface acquisition information collected by the image acquisition device;

[0007] Perform image preprocessing on the road surface acquisition information to obtain a preprocessed image;

[0008] Perform zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result;

[0009] If there is no zebra crossing image in the image zebra crossing recognition result, end the warning operation;

[0010] If the zebra crossing image exists in the image zebra crossing recognition result, obtain the current positioning information, and confirm whether there is a map zebra crossing corresponding to the current positioning information in the electronic map;

[0011] If there is no such map zebra crossing in the electronic map, end the warning operation;

[0012] If there is such map zebra crossing in the electronic map, generate a safety warning information, and output the safety warning information through the head-mounted earphone.

[0013] Further, the step of performing image preprocessing on the road surface acquisition information to obtain a preprocessed image specifically includes:

[0014] Perform binarization processing on the information collected from the road surface to obtain the preprocessed image.

[0015] Further, the step of performing zebra crossing recognition processing on the preprocessed image to obtain the image zebra crossing recognition result specifically includes:

[0016] Perform region marking processing on the preprocessed image to obtain a target region sequence, and the target region sequence includes multiple target region blocks;

[0017] Count the number of pixels for all target region blocks within the target region sequence to obtain a target marked region;

[0018] Perform rectangular judgment processing on the target marked region to obtain the zebra crossing recognition result;

[0019] Determine the image zebra crossing recognition result based on the total number of the zebra crossing recognition results.

[0020] Further, the step of performing region marking processing on the preprocessed image to obtain a target region sequence, and the target region sequence includes multiple target region blocks specifically includes:

[0021] Perform pixel traversal processing on the preprocessed image;

[0022] When a pixel point with a gray value less than the preset gray value is detected, use the pixel point with a gray value less than the preset gray value as the starting point of region marking;

[0023] Perform horizontal pixel traversal and vertical pixel traversal on the starting point of region marking until a pixel point with a gray value equal to 1 is detected, and mark the pixel point with a gray value equal to 1 as the ending point of region marking to obtain a target region block.

[0024] Further, the step of counting the number of pixels for all target region blocks within the target region sequence to obtain a target marked region specifically includes:

[0025] Perform pixel traversal on all target region blocks within the target region sequence to determine the number of pixels of each target region block;

[0026] If the number of pixels of the target region block is less than the preset minimum pixel value, determine that the target region block is an invalid region;

[0027] If the number of pixels of the target region block is greater than the preset minimum pixel value, determine that the target region block is a valid region, and determine the valid region as the target marked region.

[0028] Further, the zebra crossing recognition result includes a zebra crossing candidate area and a non-zebra crossing candidate area. The step of performing a rectangular judgment process on the target marked area to obtain the zebra crossing recognition result specifically includes:

[0029] Calculate the distances between the pixel point coordinates of the four diagonals in the target marked area;

[0030] If the distances are equal, mark the target marked area as a zebra crossing candidate area;

[0031] If the distances are not equal, mark the target marked area as a non-zebra crossing candidate area.

[0032] Further, the target recognition result includes a detection mode and a warning mode. The step of determining the image zebra crossing recognition result based on the total number of the zebra crossing recognition results specifically includes:

[0033] If there are 3 or more zebra crossing candidate areas in the target area sequence, determine that the image zebra crossing recognition result is that there is a zebra crossing image;

[0034] If the number of zebra crossing candidate areas in the target area sequence does not exceed 3, determine that the image zebra crossing recognition result is that there is no zebra crossing image.

[0035] To solve the above technical problems, an embodiment of the present application further provides a safety warning device, which adopts the following technical solutions:

[0036] The safety warning device is used for a head-mounted earphone, and the head-mounted earphone is provided with an image acquisition device. The safety warning device includes:

[0037] An acquisition module, configured to receive the road surface acquisition information collected by the image acquisition device;

[0038] A preprocessing module, configured to perform image preprocessing on the road surface acquisition information to obtain a preprocessed image;

[0039] A first processing module, configured to perform zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result;

[0040] A first judgment module, configured to end the warning operation if the image zebra crossing recognition result does not have a zebra crossing image;

[0041] A second judgment module, configured to, if the image zebra crossing recognition result has the zebra crossing image, obtain the current positioning information and confirm whether there is a map zebra crossing corresponding to the current positioning information in the electronic map;

[0042] A third judgment module, configured to end the warning operation if the map zebra crossing does not exist in the electronic map;

[0043] A fourth judgment module, configured to generate a safety warning message if the map zebra crossing exists in the electronic map, and output the safety warning message through the head-mounted earphone.

[0044] To solve the above technical problems, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:

[0045] A computer device includes a memory and a processor. Computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the steps of the above safety warning method are implemented.

[0046] To solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:

[0047] A computer-readable storage medium, characterized in that computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the steps of the above safety warning method are implemented.

[0048] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

[0049] In the present application, an image acquisition device is added to the original head-mounted earphone to capture road acquisition information. The road acquisition information is transmitted to an electronic device such as a mobile phone through Bluetooth, and then synchronized with the position information of the mobile phone and transmitted to the cloud server. The cloud server determines whether there is a zebra crossing at the location through image recognition, and can also query the map through the position information to double-check whether there is a zebra crossing at the location. Then, the information is transmitted from the cloud server to the mobile phone, and the mobile phone APP can receive the information and give a voice reminder of the traffic environment at the location, avoiding running a red light due to playing with the mobile phone with the head down, and greatly reducing the travel risk. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0051] Figure 1 It is an exemplary system architecture diagram to which the present application can be applied;

[0052] Figure 2 A flowchart according to an embodiment of the safety warning method of the present application;

[0053] Figure 3 is a schematic structural diagram of an embodiment of the security warning device according to the present application;

[0054] Figure 4 is a schematic structural diagram of an embodiment of the computer device according to the present application. Detailed implementation manners

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs; the terms used in the description of this application in the specification are only for the purpose of describing specific embodiments, and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects, not to describe a specific order.

[0056] Referring to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0057] In order to enable those skilled in the art of this technology to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0058] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a cloud server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the cloud server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0059] Users can use the terminal devices 101, 102, 103 to interact with the cloud server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0060] The terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0061] The cloud server 105 can be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103.

[0062] It should be noted that the security warning method provided by the embodiments of the present application is generally executed by the server / terminal device. Correspondingly, the security warning device is generally set in the server / terminal device.

[0063] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0064] Continue to refer to Figure 2 , which shows a flowchart of an embodiment of the security warning method according to the present application. The security warning method includes the following steps:

[0065] Step S201, receiving the road surface acquisition information collected by the image acquisition device.

[0066] In this embodiment, the electronic device (such as Figure 1 the server / terminal device shown) on which the security warning method runs can receive the request of the security warning method through a wired connection or a wireless connection. It should be noted that the above wireless connection methods can include but are not limited to 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future-developed wireless connection methods.

[0067] In an embodiment of the present invention, the above image acquisition device may be a detachable micro camera. Specifically, the above detachable micro camera can be fixed in the headphone band of the head-mounted earphone through a bracket clip. A camera interface card slot is provided in the headphone band of the above head-mounted earphone. Connecting the above detachable micro camera to the above camera interface card slot through the bracket clip can realize the device binding function between the above image acquisition device and the head-mounted earphone.

[0068] In an embodiment of the present invention, the above road surface acquisition information may be a captured image that is captured in real time by the above image acquisition device and uploaded to electronic devices such as mobile phones and tablets. Specifically, when the user wears the head-mounted earphone and executes the above safety warning method, the above detachable micro camera (i.e., the above image acquisition device) captures the front in the walking direction of the user in real time at a certain height and angle to obtain a captured image in the forward direction of the user (i.e., the above road surface acquisition information). The captured image in the forward direction of the user includes the sidewalk surface and the feet of the pedestrian. The above feet of the pedestrian refer to the sole part of the pedestrian. After the above detachable micro camera obtains the captured image in the forward direction of the user, at a frequency of transmitting one frame of the captured image every 2S, the captured image in the forward direction of the user is uploaded to electronic devices such as mobile phones and tablets through Bluetooth connection, and is uploaded to the above cloud server through the above electronic device, so that the above cloud server obtains the captured image in the forward direction of the user.

[0069] Step S202: Perform image preprocessing on the road surface acquisition information to obtain a preprocessed image.

[0070] In this embodiment, the above image preprocessing may be a process of performing a series of operations on the road surface acquisition information to improve its quality, reduce noise, highlight specific features, or simplify complexity before further processing or analysis of the road surface acquisition information. The above image preprocessing may be image denoising, image enhancement, image grayscale conversion, color space conversion, and perspective correction and other image processing measures. The specific implementation process of performing the above image preprocessing on the road surface acquisition information to obtain a preprocessed image will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated here.

[0071] Step S203: Perform zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result.

[0072] In this embodiment, the above zebra crossing recognition process can be a process of detecting and recognizing zebra crossings on the road by using an image processing method. The above image processing method can be zebra crossing detection algorithms such as the Hough Line Transform algorithm, Canny edge detection, and color threshold processing. Specifically, for the specific implementation process of performing zebra crossing recognition processing on the preprocessed image to obtain the image zebra crossing recognition result, this application will further describe the details in subsequent specific embodiments and will not elaborate too much here.

[0073] Step S204, if there is no zebra crossing image in the image zebra crossing recognition result, end the warning operation.

[0074] In this embodiment, the above warning operation can be sent to the head-mounted earphone for voice broadcast by an electronic device such as a mobile phone or a tablet through Bluetooth connection. The content of the voice broadcast can be "There is a zebra crossing ahead, please pay attention". Specifically, after the above warning operation ends, a new round of judgment on the image zebra crossing recognition result will directly start.

[0075] Step S205, if there is a zebra crossing image in the image zebra crossing recognition result, obtain the current location information and confirm whether there is a map zebra crossing corresponding to the current location information in the electronic map.

[0076] In this embodiment, the above location information can include the longitude information and latitude information of the current location. Specifically, if there is a zebra crossing image in the image zebra crossing recognition result, the user's mobile phone automatically sends the longitude information and latitude information of the current location of the mobile phone to the cloud server through the APP. After receiving the location information, the cloud server immediately calls the preset Baidu Map API interface (i.e., the above electronic map) for location query and retrieves whether there is a zebra crossing within a radius of 5m of the address obtained from the location query.

[0077] Step S206, if there is no map zebra crossing in the electronic map, end the warning operation.

[0078] In this embodiment, if there is no map zebra crossing in the electronic map, end the warning operation. The above end warning operation is the same type and same step operation as the end warning operation in step S204 and will not be elaborated too much here.

[0079] Step S207, if there is a map zebra crossing in the electronic map, generate a safety warning information and output the safety warning information through the head-mounted earphone.

[0080] In this embodiment, the above safety warning information may be preset voice information for reminding the user to pay attention to the zebra crossing ahead when walking. Specifically, the above safety warning information may be stored in the APP of electronic devices such as mobile phones and tablets, and there may be multiple types of safety warning information in one electronic device; when the cloud server meets the warning condition (that is, there is a map zebra crossing in the electronic map), the cloud server will send an electronic key to the APP of the above electronic device. The above electronic key corresponds to the above safety warning information, one electronic key corresponds to one type of safety warning information, and is sent to the head-mounted earphone for voice broadcast in a Bluetooth connection manner through the above electronic device.

[0081] In this application, an image acquisition device is added to the head-mounted earphone to capture road surface acquisition information. The road surface acquisition information is transmitted to electronic devices such as mobile phones via Bluetooth, and then synchronized with the location information of the mobile phone and transmitted to the cloud server. The cloud server determines whether there is a zebra crossing at that location through image recognition, and at the same time can query the map through the location information to double-check whether there is a zebra crossing at that location. Then, the information is transmitted to the mobile phone through the cloud server. The mobile phone APP can receive the information and give a voice reminder of the traffic environment at that location, avoiding running a red light due to looking down at the mobile phone while walking, and greatly reducing the travel risk.

[0082] In some alternative implementation manners, step S202 includes the following steps:

[0083] Perform binarization processing on the road surface acquisition information to obtain a preprocessed image.

[0084] In this embodiment, the captured image in the forward direction of the user obtained by the image acquisition device (i.e., the above road surface acquisition information) is subjected to binarization processing. Specifically, calculate the image gray value of each pixel in the captured image in the forward direction of the user, and compare it with a preset target image gray value T. Set the pixels whose detected image gray value is greater than the preset target image gray value T to 1, indicating non-white, and represent the pixels whose detected image gray value is less than the preset target image gray value T as 0, representing white. Finally, determine the binarized image as the preprocessed image.

[0085] In this embodiment, by performing binarization processing on the road surface acquisition information, the pixel points of the white zebra crossing part are distinguished, providing a good precondition for subsequent zebra crossing recognition processing.

[0086] In some alternative implementation manners, step S203 includes the following steps:

[0087] Perform region marking processing on the preprocessed image to obtain a target region sequence;

[0088] Count the number of pixels of all target region blocks in the target region sequence to obtain a target marked region;

[0089] Perform rectangular judgment processing on the target marked area to obtain the zebra crossing recognition result;

[0090] Determine the image zebra crossing recognition result based on the total number of zebra crossing recognition results.

[0091] In this embodiment, the above-mentioned area marking process may be to mark the white pixel area in the preprocessed image to obtain the target area block, and the above-mentioned target area sequence may include multiple above-mentioned target area blocks. Specifically, the specific implementation process of the above-mentioned area marking process for the preprocessed image to obtain the target area sequence will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here too much.

[0092] In this embodiment, the above-mentioned pixel quantity statistics may be to count the number of pixels in each target area block and determine the above-mentioned target marked area according to the number of pixels. Specifically, the specific implementation process of the above-mentioned pixel quantity statistics for all target area blocks in the target area sequence to obtain the target marked area will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here too much.

[0093] In this embodiment, the specific implementation process of the above-mentioned rectangular judgment processing on the target marked area to obtain the zebra crossing recognition result will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here too much.

[0094] In this embodiment, the specific implementation process of the above-mentioned determining the image zebra crossing recognition result based on the total number of zebra crossing recognition results will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here too much.

[0095] This embodiment extracts the target area sequence in the image through area marking processing, then determines the target marked area by using pixel quantity statistics, filters out the areas that conform to the rectangular shape through rectangular judgment processing, and finally obtains the zebra crossing recognition result, providing an efficient automatic image processing and recognition basis for zebra crossing recognition, and can improve the accuracy and efficiency of zebra crossing recognition.

[0096] In some alternative implementation manners, the step "perform area marking processing on the preprocessed image to obtain the target area sequence" includes the following steps:

[0097] Perform pixel traversal processing on the preprocessed image;

[0098] When a pixel point smaller than the preset gray value is detected, the pixel point smaller than the preset gray value is used as the starting point of area marking;

[0099] Perform a horizontal pixel traversal and a vertical pixel traversal on the starting point of the region marker until a pixel point with a gray value equal to 1 is detected, and mark the pixel point with a gray value equal to 1 as the end point of the region marker to obtain the target region block.

[0100] In this embodiment, for the above-mentioned vertical pixel traversal, assuming that the coordinates of the starting point of the region marker are (x, y), perform vertical traversal in the form of (x, y + 1) until the pixel point of (x, y + 1) satisfies the vertical traversal stop condition. The above-mentioned vertical traversal stop condition can be that the gray value of the pixel point of (x, y + 1) is equal to 1 or the pixel point of (x, y + 1) is on the vertical image boundary of the above-mentioned preprocessed image. When the above-mentioned vertical traversal stop condition is satisfied, continue the vertical pixel traversal at the pixel point (x + 1, y) of the horizontal pixel traversal until (x + 1, y) satisfies the horizontal traversal stop condition to obtain the end point of the region marker. The above-mentioned horizontal traversal stop condition can be that the gray value of the pixel point of (x + 1, y) is equal to 1 or the pixel point of (x, y + 1) is on the horizontal image boundary of the above-mentioned preprocessed image.

[0101] In a possible embodiment, for the above-mentioned horizontal pixel traversal, perform horizontal traversal in the form of (x + 1, y) until the pixel point of (x + 1, y) satisfies the horizontal traversal stop condition. When the above-mentioned horizontal traversal stop condition is satisfied, continue the horizontal pixel traversal at the pixel point (x, y + 1) of the vertical pixel traversal until (x, y + 1) satisfies the horizontal traversal stop condition to obtain the end point of the region marker.

[0102] In this embodiment, start traversing from the point with coordinates (0, 0) of the above-mentioned preprocessed image (i.e., pixel traversal processing). If it is found that the gray value of the point (x, y) is less than the preset gray value T, then mark this point as the starting point of the module white area, and then traverse in two directions of (x + 1, y) and (x, y + 1) (i.e., the above-mentioned horizontal pixel traversal and vertical pixel traversal). If the gray value of one of the pixel points (x + 1, y) and (x, y + 1) is less than 0, it means that one or both of these adjacent points are white points, and continue to iterate and search. If it is detected that the gray value of a certain pixel point is equal to 1, it means that there are no white points in the adjacent area of this point. Mark the pixel point with a gray value equal to 1 as the end point of the region marker to obtain the target region block, and restart the above-mentioned pixel traversal processing with the above-mentioned end point of the region marker as the starting point until all pixel points of the above-mentioned preprocessed image are traversed.

[0103] This application divides the preprocessed image into small blocks so that each block contains a target region. By performing horizontal and vertical pixel traversals, the exact range of the target region can also be obtained, and irregularly shaped regions can also be processed, not just limited to rectangular or square regions, which helps to perform further processing on specific regions.

[0104] In some alternative implementations, the step of "counting the number of pixels in all target region blocks within the target region sequence to obtain a target marked region" includes the following steps:

[0105] Traverse the pixels in all target region blocks within the target region sequence to determine the number of pixels in each target region block;

[0106] If the number of pixels in a target region block is less than a preset minimum pixel value, determine that the target region block is an invalid region;

[0107] If the number of pixels in a target region block is greater than the preset minimum pixel value, determine that the target region block is a valid region and identify the valid region as the target marked region.

[0108] In this embodiment, the minimum number of pixels in a target region block is the preset minimum pixel value, and only then can a white zebra crossing be recognized in subsequent zebra crossing recognition processing. Specifically, if the number of pixels in a target region block is less than the preset minimum pixel value, it indicates that the above target region block is an invalid region and is directly discarded; if the number of pixels in a target region block is greater than the preset minimum pixel value, it is determined that the target region block is a valid region, and the valid region is identified as the target marked region, and all the pixels in the above target marked region are white pixels.

[0109] By setting the minimum pixel value in this application, noise and small-area target region blocks can be effectively filtered out, reducing the processing of these unimportant regions and improving the robustness of the algorithm.

[0110] In some alternative implementations, the step of "performing rectangular judgment processing on the target marked region to obtain a zebra crossing recognition result" includes the following steps:

[0111] Calculate the distances between the pixel coordinates of the four diagonals in the target marked region;

[0112] If the distances are equal, mark the target marked region as a zebra crossing candidate region;

[0113] If the distances are not equal, mark the target marked region as a non-zebra crossing candidate region.

[0114] In this embodiment, the pixel point coordinates (x, y), (x, ymax), (xmax, y), and (xmax, ymax) of the four diagonals in the above-mentioned target marked area are counted, and it is calculated whether the value of (xmax, ymax)-(x, ymax) is equal to the value of (xmax, y)-(x, y), and whether the value of (x, ymax)-(x, y) is equal to the value of (xmax, ymax)-(xmax, y). If the distances between the two are equal, the target marked area is marked as a zebra crossing candidate area; if the distances between the two are not equal, the target marked area is marked as a non-zebra crossing candidate area.

[0115] In this embodiment, the above-mentioned zebra crossing candidate area is a target marked area that meets the zebra crossing characteristics, and the above-mentioned zebra crossing characteristics may be equal distance characteristics on the diagonal. The above-mentioned non-zebra crossing candidate area may be a target marked area with an irregular shape.

[0116] By calculating the distances between the pixel point coordinates of the four diagonals in the target marked area, this application can obtain more accurate distance information, thereby improving the determination accuracy of the zebra crossing candidate area.

[0117] In some optional implementation manners, the step of "determining the image zebra crossing recognition result based on the total number of the zebra crossing recognition results" includes the following steps:

[0118] If there are 3 or more zebra crossing candidate areas in the target area sequence, it is determined that the image zebra crossing recognition result is that there is a zebra crossing image;

[0119] If the number of zebra crossing candidate areas in the target area sequence does not exceed 3, it is determined that the image zebra crossing recognition result is that there is no zebra crossing image.

[0120] In this embodiment, by determining the total number of zebra crossing candidate areas in the target area sequence to judge whether there is a zebra crossing image in the image zebra crossing recognition result, the judgment method directly based on the number of candidate areas is relatively simple, which can speed up the image processing speed, reduce the possibility of misjudgment, and improve the judgment accuracy.

[0121] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above method embodiments. Among them, the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0122] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction for the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0123] Further referring to Figure 3 and as an implementation of the method shown above Figure 2 , an embodiment of a security warning device is provided in this application. This device embodiment corresponds to the method embodiment shown in Figure 2 , and this device can be specifically applied to various electronic devices.

[0124] As shown in Figure 3 , the security warning device 300 described in this embodiment includes: an acquisition module 301, a preprocessing module 302, a first processing module 303, a first judgment module 304, a second judgment module 305, a third judgment module 306, and a fourth judgment module 307. Among them:

[0125] The acquisition module 301 is configured to receive the road surface acquisition information collected by the image acquisition device;

[0126] The preprocessing module 302 is configured to perform image preprocessing on the road surface acquisition information to obtain a preprocessed image;

[0127] The first processing module 303 is configured to perform zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result;

[0128] The first judgment module 304 is configured to end the warning operation if there is no zebra crossing image in the image zebra crossing recognition result;

[0129] The second judgment module 305 is configured to obtain the current positioning information if there is the zebra crossing image in the image zebra crossing recognition result, and confirm whether there is a map zebra crossing corresponding to the current positioning information in the electronic map;

[0130] The third judgment module 306 is configured to end the warning operation if there is no such map zebra crossing in the electronic map;

[0131] The fourth judgment module 307 is configured to generate a safety warning message if there is the map zebra crossing in the electronic map, and output the safety warning message through the head-mounted earphone.

[0132] In this embodiment, an image acquisition device is added to the original headphone to capture road surface acquisition information. The road surface acquisition information is transmitted to an electronic device such as a mobile phone through Bluetooth, and then is synchronously transmitted to the cloud server in combination with the position information of the mobile phone. The cloud server determines whether there is a zebra crossing at the place through image recognition, and at the same time, it can query the map through the position information to double-check whether there is a zebra crossing at the place. Then, the information is transmitted to the mobile phone by the cloud server. The mobile phone APP can receive the information and give a voice reminder of the traffic environment at the place, avoiding running a red light because of lowering the head to play with the mobile phone, and greatly reducing the travel risk.

[0133] In some optional implementation manners of this embodiment, the preprocessing module 302 further includes a first processing sub-module, where:

[0134] The first processing sub-module is configured to perform binarization processing on the road surface acquisition information to obtain the preprocessed image.

[0135] In this embodiment, by performing binarization processing on the road surface acquisition information, the pixel points of the white zebra crossing part are distinguished, providing a good precondition for subsequent zebra crossing recognition processing.

[0136] In some optional implementation manners of this embodiment, the first processing module 303 further includes a marking sub-module, a statistical sub-module, an identification sub-module, and a determination sub-module, where:

[0137] The marking sub-module is configured to perform region marking processing on the preprocessed image to obtain a target region sequence, and the target region sequence includes a plurality of target region blocks;

[0138] The statistical sub-module is configured to perform pixel quantity statistics on all target region blocks in the target region sequence to obtain a target marked region;

[0139] The identification sub-module is configured to perform rectangular judgment processing on the target marked region to obtain a zebra crossing recognition result;

[0140] The determination sub-module is configured to determine the image zebra crossing recognition result based on the total number of the zebra crossing recognition results.

[0141] In this embodiment, the target region sequence in the image is extracted through region marking processing, and then the target marked region is determined by using pixel quantity statistics. The regions that conform to the rectangular shape are screened out through rectangular judgment processing, and finally the zebra crossing recognition result is obtained, providing an efficient automatic image processing and recognition basis for identifying the zebra crossing, and can improve the accuracy and efficiency of zebra crossing recognition.

[0142] In some alternative implementation manners of this embodiment, the marking sub-module further includes a first traversal unit, a first marking unit, and a second marking unit, where:

[0143] The first traversal unit is configured to perform pixel traversal processing on the preprocessed image;

[0144] The first marking unit is configured to use the pixel point smaller than the preset gray value as the starting point of area marking when detecting a pixel point smaller than the preset gray value;

[0145] The second marking unit is configured to perform horizontal pixel traversal and vertical pixel traversal on the starting point of area marking until detecting a pixel point with a gray value equal to 1, and mark the pixel point with a gray value equal to 1 as the ending point of area marking to obtain a target area block.

[0146] In this application, by dividing the preprocessed image into small blocks, each block contains a target area. By performing horizontal and vertical pixel traversals, the exact range of the target area can be obtained, and irregularly shaped areas can also be processed, not just limited to rectangular or square areas, which helps to perform further processing on specific areas.

[0147] In some alternative implementation manners of this embodiment, the statistical sub-module further includes a second traversal unit, a first determination unit, and a second determination unit, where:

[0148] The second traversal unit is configured to perform pixel traversal on all target area blocks in the target area sequence to determine the number of pixels in each target area block;

[0149] The first determination unit is configured to determine that the target area block is an invalid area if the number of pixels in the target area block is less than the preset minimum pixel value;

[0150] The second determination unit is configured to determine that the target area block is a valid area if the number of pixels in the target area block is greater than the preset minimum pixel value, and determine the valid area as the target marking area.

[0151] In this application, by setting the minimum pixel value, noise and small-area target area blocks can be effectively filtered out, reducing the processing of these unimportant areas and improving the robustness of the algorithm.

[0152] In some alternative implementation manners of this embodiment, the recognition sub-module further includes a calculation unit, a third marking unit, and a fourth marking unit, where:

[0153] The calculation unit is configured to calculate the distances between the pixel point coordinates of four diagonals in the target marking area;

[0154] A third marking unit, configured to mark the target marked area as a zebra crossing candidate area if the distances are equal;

[0155] A fourth marking unit, configured to mark the target marked area as a non - zebra crossing candidate area if the distances are not equal.

[0156] By calculating the distances between the pixel point coordinates of the four diagonals in the target marked area, the present application can obtain more accurate distance information, thereby improving the determination accuracy of the zebra crossing candidate area.

[0157] In some alternative implementation manners of this embodiment, the determination sub - module further includes a first determination unit and a second determination unit, where:

[0158] The first determination unit is configured to determine that the image zebra crossing recognition result is that there is a zebra crossing image if there are 3 or more of the zebra crossing candidate areas in the target area sequence;

[0159] The second determination unit is configured to determine that the image zebra crossing recognition result is that there is no zebra crossing image if the number of the zebra crossing candidate areas in the target area sequence does not exceed 3.

[0160] In this embodiment, by determining the total number of zebra crossing candidate areas in the target area sequence to judge whether there is a zebra crossing image in the image zebra crossing recognition result, the judgment method directly based on the number of candidate areas is relatively simple, can speed up the image processing speed, can reduce the possibility of misjudgment, and improves the judgment accuracy.

[0161] To solve the above - mentioned technical problems, the embodiment of the present application further provides a computer device. For details, please refer to Figure 4 , Figure 4 which is the basic structural block diagram of the computer device in this embodiment.

[0162] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are communicatively connected to each other via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0163] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, a voice control device, etc.

[0164] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the security warning method. In addition, the memory 41 can also be used to temporarily store various data that have been output or will be output.

[0165] In some embodiments, the processor 42 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions of the security warning method.

[0166] The network interface 43 may include a wireless network interface or a wired network interface, which is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0167] This application provides a computer device. By adding an image acquisition device to the head-mounted earphone in this application to capture road acquisition information, the road acquisition information is transmitted to electronic devices such as mobile phones via Bluetooth, and then combined with the location information of the mobile phone and synchronously transmitted to the cloud server. The cloud server determines whether there is a zebra crossing at that location through image recognition, and at the same time can query the map through the location information to double-check whether there is a zebra crossing at that location. Then, the information is transmitted to the mobile phone through the cloud server. The mobile phone APP receives the information and can voice-remind the traffic environment at that location, avoiding running a red light due to playing with the mobile phone with the head down, and greatly reducing the travel risk.

[0168] This application also provides another implementation manner, that is, to provide a computer-readable storage medium. The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to execute the steps of the security warning method as described above.

[0169] This application provides a computer-readable storage medium. By adding an image acquisition device to the head-mounted earphone in this application to capture road acquisition information, the road acquisition information is transmitted to electronic devices such as mobile phones and tablets via Bluetooth, and then combined with the location information of the mobile phone and synchronously transmitted to the cloud server. The cloud server determines whether there is a zebra crossing at that location through image recognition, and at the same time can query the map through the location information to double-check whether there is a zebra crossing at that location. Then, the information is transmitted to the mobile phone through the cloud server. The mobile phone APP receives the information and can voice-remind the traffic environment at that location, avoiding running a red light due to playing with the mobile phone with the head down, and greatly reducing the travel risk.

[0170] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0171] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields is equally within the scope of the patent protection of the present application.

Claims

1. A safety warning method, applicable to a head-mounted earphone, characterized in that, The head-mounted earphone is provided with an image acquisition device, including the following steps: Receiving the road surface acquisition information collected by the image acquisition device; Performing image preprocessing on the road surface acquisition information to obtain a preprocessed image; Performing zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result; If there is no zebra crossing image in the image zebra crossing recognition result, end the warning operation; If the zebra crossing image exists in the image zebra crossing recognition result, obtain the current positioning information and confirm whether there is a map zebra crossing corresponding to the current positioning information in the electronic map; If there is no such map zebra crossing in the electronic map, end the warning operation; If there is such map zebra crossing in the electronic map, generate a safety warning information and output the safety warning information through the head-mounted earphone.

2. The safety warning method according to claim 1, wherein, The step of performing image preprocessing on the road surface acquisition information to obtain a preprocessed image specifically includes: Performing binarization processing on the road surface acquisition information to obtain the preprocessed image.

3. The safety warning method according to claim 1, characterized in that, The step of performing zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result specifically includes: Performing region marking processing on the preprocessed image to obtain a target region sequence, and the target region sequence includes a plurality of target region blocks; Performing pixel quantity statistics on all target region blocks within the target region sequence to obtain a target marked region; Performing rectangular judgment processing on the target marked region to obtain a zebra crossing recognition result; Determining the image zebra crossing recognition result based on the total number of the zebra crossing recognition results.

4. The safety warning method according to claim 3, characterized in that, The step of performing region marking processing on the preprocessed image to obtain a target region sequence, and the target region sequence includes a plurality of target region blocks specifically includes: Performing pixel traversal processing on the preprocessed image; If a pixel point with a gray value less than a preset gray value is detected, use the pixel point with a gray value less than the preset gray value as the region marking starting point; Performing horizontal pixel traversal and vertical pixel traversal on the region marking starting point until a pixel point with a gray value equal to 1 is detected, and marking the pixel point with a gray value equal to 1 as the region marking ending point to obtain a target region block.

5. The safety warning method according to claim 3, wherein The step of performing pixel quantity statistics on all target region blocks within the target region sequence to obtain a target marked region specifically includes: Performing pixel traversal on all target region blocks within the target region sequence to determine the pixel quantity of each target region block; If the pixel quantity of the target region block is less than a preset minimum pixel value, determine that the target region block is an invalid region; If the pixel quantity of the target region block is greater than a preset minimum pixel value, determine that the target region block is a valid region, and determine the valid region as the target marked region.

6. The safety warning method according to claim 5, characterized in that, The zebra crossing recognition result includes a zebra crossing candidate region and a non-zebra crossing candidate region. The step of performing rectangular judgment processing on the target marked region to obtain a zebra crossing recognition result specifically includes: Calculating the distances between the pixel point coordinates of the four diagonals in the target marked region; If the distances are equal, mark the target marked region as a zebra crossing candidate region; If the distances are not equal, mark the target marked area as a non-zebra crossing candidate area.

7. The safety warning method according to claim 6, wherein The target recognition result includes a detection mode and a warning mode. The step of determining the image zebra crossing recognition result based on the total number of the zebra crossing recognition results specifically includes: If there are 3 or more of the zebra crossing candidate areas in the target area sequence, determine that the image zebra crossing recognition result is that there is a zebra crossing image; If the number of the zebra crossing candidate areas in the target area sequence does not exceed 3, determine that the image zebra crossing recognition result is that there is no zebra crossing image.

8. A safety warning device, characterized in that, The safety warning device is used for a head-mounted earphone, and the head-mounted earphone is provided with an image acquisition device. The safety warning device includes: An acquisition module, configured to receive the road surface acquisition information collected by the image acquisition device; A preprocessing module, configured to perform image preprocessing on the road surface acquisition information to obtain a preprocessed image; A first processing module, configured to perform zebra crossing recognition processing on the preprocessed image to obtain an image zebra crossing recognition result; A first judgment module, configured to end the warning operation if the image zebra crossing recognition result is that there is no zebra crossing image; A second judgment module, configured to, if the image zebra crossing recognition result is that there is the zebra crossing image, obtain the current positioning information and confirm whether there is a map zebra crossing corresponding to the current positioning information in the electronic map; A third judgment module, configured to end the warning operation if the map zebra crossing does not exist in the electronic map; A fourth judgment module, configured to, if the map zebra crossing exists in the electronic map, generate a safety warning information and output the safety warning information through the head-mounted earphone.

9. A computer device, including a memory and a processor, where computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the steps of the safety warning method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that Computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by the processor, the steps of the safety warning method according to any one of claims 1 to 7 are implemented.

Images