Collision warning method, device, head-mounted device, head-mounted system and medium

Abstract

The present application discloses a collision warning method, apparatus, head-mounted device, head-mounted system, and medium, relating to the field of head-mounted technology. The method is applied to a head-mounted device equipped with at least one handle, and comprises: obtaining a current brightness change image captured by an event camera provided on the handle at the current moment; determining, based on the current brightness change image, the positional relationship between the handle, the head-mounted device, and obstacles in the environment in which the head-mounted device is located; determining, based on the positional relationship, whether the wearer of the head-mounted device is at risk of collision; and, if so, outputting a first collision warning prompt message. This method prevents collisions from occurring while the wearer is wearing the head-mounted device.

Description

Technical Field

[0001] The present application relates to the field of head-mounted technology, and more specifically, to a collision warning method, a collision warning device, a head-mounted device, a head-mounted system, and a computer-readable storage medium. Background Art

[0002] With the development of science and technology and economy, head-mounted devices (such as VR / AR / MR helmets or glasses) have become more and more widely used. Currently, most head-mounted devices are equipped with handles.

[0003] When a user wears a head-mounted device, their perception of the surrounding environment is often impaired due to the device's obstruction. This creates a risk of collision with objects in the user's environment while using the device. When a collision occurs, the user's experience is significantly reduced. Therefore, preventing collisions between users and objects while wearing a head-mounted device has become a pressing technical issue. Summary of the Invention

[0004] One purpose of this application is to provide a new technical solution for collision warning.

[0005] According to a first aspect of the present application, a collision warning method is provided, which is applied to a head-mounted device, wherein the head-mounted device is adapted to be equipped with at least one handle, and the method comprises:

[0006] Obtaining a current brightness change image captured by the event camera set on the handle at the current moment;

[0007] determining, based on the current brightness change image, a positional relationship among the handle, the head mounted device, and obstacles in an environment in which the head mounted device is located;

[0008] determining, based on the positional relationship, whether a wearer of the head mounted device is at risk of collision;

[0009] If so, a first collision warning prompt message is output.

[0010] Optionally, determining whether the wearer of the head mounted device is at risk of collision based on the positional relationship includes:

[0011] Determine, based on the current brightness change image, spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point, where the first pixel point is a pixel point corresponding to an obstacle in the environment, and the second pixel point is a pixel point corresponding to the wearer;

[0012] Determining a first distance and a second distance based on the spatial information, the first pixel point, and the second pixel point, where the first distance is a distance between the handle and an obstacle in the environment where the handle is located, and the second distance is a distance between the handle and a wearer of the head mounted device;

[0013] When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, it is determined that the wearer of the head mounted device is at risk of collision.

[0014] Optionally, after obtaining the current brightness change image captured by the event camera provided on the handle at the current moment, the method further includes:

[0015] Acquire at least one frame of historical brightness change image;

[0016] determining, based on the historical brightness change image and the current brightness change image, whether there is a moving obstacle in the environment moving toward the wearer;

[0017] If so, a second collision warning prompt message is output.

[0018] Optionally, before determining the positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located based on the current brightness change image, the method further includes:

[0019] In a case where the current brightness change image consists of at least two frames, the at least two frames of the current brightness change image are fused into one frame of the current brightness change image.

[0020] Optionally, determining, based on the current brightness change image, a positional relationship among the handle, the head mounted device, and obstacles in an environment in which the head mounted device is located includes:

[0021] filtering out noise in the current brightness change image;

[0022] The positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located is determined based on the current brightness change image after noise is filtered out.

[0023] According to a second aspect of the present application, a collision warning device is provided, which is applied to a head-mounted device, wherein the head-mounted device is adapted to be equipped with at least one handle, and the device includes:

[0024] An acquisition module, configured to acquire a current brightness change image captured by an event camera provided on the handle at a current moment;

[0025] a first determining module, configured to determine a positional relationship among the handle, the head mounted device, and obstacles in an environment where the head mounted device is located based on the current brightness change image;

[0026] a second determining module, configured to determine, based on the positional relationship, whether the wearer of the head mounted device is at risk of collision;

[0027] The warning module is used to output a first collision warning prompt message when there is a collision.

[0028] Optionally, the second determining module is specifically configured to:

[0029] Determine, based on the current brightness change image, spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point, where the first pixel point is a pixel point corresponding to an obstacle in the environment, and the second pixel point is a pixel point corresponding to the wearer;

[0030] Determining a first distance and a second distance based on the spatial information, the first pixel point, and the second pixel point, where the first distance is a distance between the handle and an obstacle in the environment where the handle is located, and the second distance is a distance between the handle and a wearer of the head mounted device;

[0031] When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, it is determined that the wearer of the head mounted device is at risk of collision.

[0032] According to a third aspect of the present application, a head-mounted device is provided, characterized in that the head-mounted device includes the device as described in any one of the second aspects; or,

[0033] It comprises a memory and a processor, the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the collision warning method as described in any one of the first aspects.

[0034] According to a fourth aspect of the present application, a head-mounted system is provided, comprising the head-mounted device as described in the third aspect and at least one handle, any of the handles being provided with at least one event camera.

[0035] According to a fifth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the collision warning method according to any one of the first aspects.

[0036] An embodiment of the present application provides a collision warning method, which is applied to a head-mounted device, wherein the head-mounted device is adapted to be equipped with at least one handle. The method comprises: obtaining a current brightness change image captured by an event camera provided on the handle at the current moment; determining the positional relationship between the handle, the head-mounted device, and obstacles in the environment in which the head-mounted device is located based on the current brightness change image; determining whether the wearer of the head-mounted device is at risk of collision based on the positional relationship; and outputting a first collision warning prompt message if the wearer is at risk of collision through the collision warning method provided by the embodiment of the present application. In this way, collisions can be avoided while the wearer is wearing the head-mounted device.

[0037] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0039] Figure 1 This is a block diagram of the hardware configuration of a head-mounted device for implementing a collision warning method according to an embodiment of the present application;

[0040] Figure 2 This is a flowchart of a method for implementing a collision warning according to an embodiment of the present application;

[0041] Figure 3 is a structural diagram of a collision warning device provided according to an embodiment of the present application;

[0042] Figure 4 This is a structural diagram of a head-mounted device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0044] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0045] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0046] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0047] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] Figure 1 This is a block diagram of the hardware configuration of a head-mounted device for implementing a collision warning method provided in an embodiment of the present application.

[0049] The head mounted device 1000 may be, for example, an AR device, an MR device, or a VR device. Furthermore, the head mounted device 1000 is adapted to be equipped with at least one handle, and the handle is provided with at least one event camera.

[0050] The head-mounted device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a central processing unit (CPU), a microprocessor (MCU), or the like. The memory 1200 may include, for example, ROM (read-only memory), RAM (random access memory), or a non-volatile memory such as a hard disk. The interface device 1300 may include, for example, a USB port or a headphone jack. The communication device 1400 may be capable of wired or wireless communication. The display device 1500 may be, for example, an LCD display or a touchscreen display. The input device 1600 may include, for example, a touchscreen or a keyboard. The user may input and output voice information through the speaker 1700 and microphone 1800.

[0051] Despite Figure 1 Multiple devices are shown for the head-mounted device 1000, but the present application may only involve some of the devices, for example, the head-mounted device 1000 only involves the memory 1200 and the processor 1100.

[0052] In the embodiment of the present application, the memory 1200 of the head mounted device 1000 is used to store instructions, which are used to control the processor 1100 to execute the collision warning method provided in the embodiment of the present application.

[0053] In the above description, a person skilled in the art can design instructions according to the solution disclosed in this application. How instructions control the operation of a processor is well known in the art and will not be described in detail here.

[0054] The embodiment of the present application provides a collision warning method, such as Figure 2As shown, the method includes the following steps S2100-S2400:

[0055] S2100: Obtain a current brightness change image captured by an event camera set on the handle at the current moment.

[0056] The collision warning method provided in the embodiment of the present application is applied to a head-mounted device, which is adapted to be equipped with at least one handle.

[0057] In an embodiment of the present application, at least one event camera is provided on the handle. The image acquisition direction of the event camera includes the direction of the environment in which the head-mounted device and the handle (or the head-mounted device) are located.

[0058] Among them, the event camera is a sensor that perceives moving objects. Each pixel of the event camera has an independent photoelectric sensing module. When the brightness change at the pixel exceeds the set threshold, event data will be generated and output. The event data includes the pixel coordinates, timestamp and polarity information of the pixel point whose brightness change value is greater than the threshold C. The polarity information is used to characterize whether the brightness of the corresponding pixel point is enhanced or weakened. C can be set based on experience, for example, set to 10% to 50% of the brightness. It should be noted that in the embodiment of the present application, the event data output by the event camera at the current moment is stacked together according to the pixel position to form an image as the current brightness change image.

[0059] Since the handle moves during use, the event camera also moves when the handle moves. At this point, the edges and background of objects in the event camera's environment experience significant brightness variations. Therefore, the current brightness variation image obtained by the event camera can reflect information such as the outline, posture, and position of objects in the handle's environment. The objects in the handle's environment include at least the wearer, the headset, the handle, and other objects. It is understandable that the remaining objects are obstacles relative to the wearer.

[0060] It is understandable that, when a handle is provided on the head mounted device and an event camera is provided on the handle, only one frame of the current brightness change image can be obtained based on the above S2100.

[0061] When two handles are provided on the head mounted device and at least one event camera is provided on each handle, at least two frames of current brightness change images can be obtained based on the above S2100.

[0062] S2200: Determine, based on the current brightness change image, the positional relationship among the handle, the head-mounted device, and obstacles in the environment in which the head-mounted device is located.

[0063] In an embodiment of the present application, the event camera on the handle can capture images of the head-mounted device and obstacles in the environment in which the head-mounted device is located. By performing image analysis on the images captured by the handle, the coordinate information of the obstacle and the head-mounted device relative to the handle can be obtained. Furthermore, based on the coordinate information of the obstacle and the head-mounted device relative to the handle, the positional relationship between the handle, the head-mounted device, and the obstacle in the environment in which the head-mounted device is located can be determined. The positional relationship includes but is not limited to at least one of the distance, the angle, and the triangle formed between the three. It can be understood that the triangle can reflect the distance and angle between the handle, the head-mounted device, and the obstacle in the environment in which the head-mounted device is located.

[0064] Furthermore, the headset is typically equipped with a camera for image acquisition. Based on this, the headset can combine the current brightness change image with the image captured by the headset's camera to perform image analysis, thereby determining the positional relationship between the handle, the headset, and obstacles in the environment in which the headset is located.

[0065] It should be noted that the wearer wears a head-mounted device, so the position of the head-mounted device can also represent the position of the wearer. In addition, the embodiment of the present application does not limit the specific implementation of the above S2200.

[0066] S2300: Determine whether the wearer of the head mounted device is at risk of collision based on the positional relationship.

[0067] In the embodiment of the present application, whether a collision occurs specifically refers to whether the wearer collides with an obstacle.

[0068] In one embodiment of the present application, when the positional relationship is specifically the distance between the handle, the wearer of the head mounted device, and obstacles in the environment where the head mounted device is located, the above S2300 may be specifically implemented as follows S2310 to S2312:

[0069] S2310: Determine spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point according to the current brightness change image.

[0070] Among them, the first pixel point is the pixel point corresponding to the obstacle in the environment, and the second pixel point is the pixel point corresponding to the wearer.

[0071] In the embodiment of the present application, the spatial information of the environment in which the handle is located specifically refers to the three-dimensional coordinate positions of the feature points of the objects constituting the environment relative to the handle.

[0072] In one embodiment of the present application, the spatial information of the environment in which the controller is located can be calculated based on the current brightness change image, the calibration information of the event camera, and a monocular depth estimation algorithm. The calibration information of the event camera can be obtained in advance through traditional camera calibration methods and pre-stored in the head-mounted device.

[0073] Furthermore, pixels belonging to the wearer in the current brightness changing image can be identified using a feature point detection and tracking algorithm, a combined clustering algorithm, or the like. Based on this, the identified pixels belonging to the wearer in the current brightness changing image are recorded as second pixels. Similarly, pixels belonging to obstacles in the current brightness changing image are identified and recorded as first pixels.

[0074] It is understandable that there are multiple first pixel points and multiple second pixel points.

[0075] S2311. Determine a first distance and a second distance according to the spatial information, the first pixel point, and the second pixel point.

[0076] The first distance is the distance between the handle and an obstacle in the environment in which the handle is located. The second distance is the distance between the handle and the wearer of the head mounted device.

[0077] In the embodiment of the present application, first, a first feature point corresponding to the first pixel point is found from the spatial information, and a second feature point corresponding to the second pixel point is found.

[0078] Combined with the spatial information, the three-dimensional coordinate position of the first feature point relative to the event camera can be determined, and further based on the three-dimensional coordinate position of the first feature point relative to the event camera, the first distance can be obtained.

[0079] Similarly, combined with the spatial information, the three-dimensional coordinate position of the second feature point relative to the event camera can be determined, and further based on the three-dimensional coordinate position of the second feature point relative to the event camera, the second distance can be obtained.

[0080] S2312: When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, determine that the wearer of the head mounted device is at risk of collision.

[0081] Among them, the preset threshold is the maximum distance value between the wearer of the head-mounted device and the obstacle when the wearer is at risk of collision.

[0082] In the embodiment of the present application, combined with the description of the first distance and the second distance described above, the difference between the first distance and the second distance can represent the distance between the wearer and the obstacle. If the difference between the first distance and the second distance is less than or equal to a preset threshold, it indicates that the wearer is at risk of collision. At the same time, if the distance between the wearer and the obstacle is also less than or equal to the preset threshold, it also indicates that the wearer is at risk of collision. Therefore, based on the dual determination in S2312 above, it can be accurately determined that the wearer of the head-mounted device is at risk of collision.

[0083] Corresponding to the above S2312, when the difference between the first distance and the second distance, or the distance between the wearer and the obstacle is greater than the preset threshold, it means that the wearer is not at risk of collision.

[0084] In another embodiment of the present application, in the case where the positional relationship is the angle between the handle, the headset, and the obstacle, if the angle between the straight line formed between the handle and the obstacle and the straight line formed between the headset and the obstacle is an obtuse angle, it means that the wearer is far away from the obstacle and there is no collision risk for the wearer. Conversely, it means that the wearer is close to the obstacle and there is a collision risk for the wearer.

[0085] Alternatively, the above-mentioned angle corresponding to the current brightness change image is compared with the above-mentioned angle corresponding to the brightness change image in the previous frame of the current brightness change image. If the angle becomes larger, it means that the obstacle is closer to the wearer, that is, there is a trend of approaching between the obstacle and the wearer. At this time, if the distance between the obstacle and the wearer is less than or equal to the preset threshold, the wearer is at risk of collision.

[0086] It's understood that the distance between the handle and the headset is typically within a fixed range. Therefore, the larger the angle, the closer the obstacle is to the headset. Conversely, the smaller the angle, the farther the obstacle is from the headset.

[0087] Of course, the above S2300 can also be implemented in other ways. Any specific implementation method of determining whether the wearer of the head-mounted device is at risk of collision based on the positional relationship between the handle, the head-mounted device, and the obstacle is within the scope of protection of this application.

[0088] S2400: If the collision exists, output a first collision warning prompt message.

[0089] In an embodiment of the present application, when the wearer is at risk of collision, a first collision warning prompt message is output to remind the wearer of the risk of collision. The first collision warning prompt message can be a prompt message based on at least one of sound, light, text, image, and vibration.

[0090] Based on the first collision warning prompt information, the wearer can adjust their movement direction, etc., to avoid collision with obstacles. In other words, the collision warning method provided by the embodiments of the present application can alert the wearer when there is a risk of collision. In this way, collisions can be avoided while the wearer is wearing the head-mounted device.

[0091] Furthermore, in the embodiments of the present application, only an event camera is required on the handle to describe the outline, posture, and position of objects in the environment in which the handle is located, eliminating the need for at least four cameras and an IMU unit for motion compensation on the head-mounted device. Therefore, the hardware cost of implementing the collision warning method provided by the embodiments of the present application is low.

[0092] Corresponding to the above S2400 , when there is no collision risk, the above S2100 is repeatedly executed.

[0093] An embodiment of the present application provides a collision warning method, which is applied to a head-mounted device, wherein the head-mounted device is adapted to be equipped with at least one handle. The method comprises: obtaining a current brightness change image captured by an event camera provided on the handle at the current moment; determining the positional relationship between the handle, the head-mounted device, and obstacles in the environment in which the head-mounted device is located based on the current brightness change image; determining whether the wearer of the head-mounted device is at risk of collision based on the positional relationship; and outputting a first collision warning prompt message if the wearer is at risk of collision through the collision warning method provided by the embodiment of the present application. In this way, collisions can be avoided while the wearer is wearing the head-mounted device.

[0094] In one embodiment of the present application, the collision warning method provided in the embodiment of the present application further includes the following steps S2500 to S2700 after the above S2100:

[0095] S2500: Acquire at least one frame of historical brightness change image.

[0096] In the embodiment of the present application, the historical brightness change image is a current brightness change image captured during at least one execution of the collision warning method provided by the embodiment of the present application before the current moment. For example, the historical brightness change images are the current brightness change images captured during the last and the previous execution of the collision warning method provided by the embodiment of the present application.

[0097] It should be noted that, in the embodiment of the present application, the number of frames of the historical brightness change image is not limited and can be set based on experience, for example, the number of frames of the historical brightness change image is set to 3 frames.

[0098] S2600: Determine whether there is a moving obstacle moving toward the wearer in the environment based on the historical brightness change image and the current brightness change image.

[0099] In an embodiment of the present application, the specific implementation of the above S2600 may be: determining the speed of an object in the environment based on a historical brightness change image, a current brightness change image, and a Kalman filter. For objects with a speed greater than 0, image recognition is performed to identify objects other than the wearer and the head-mounted device. The identified object is determined to be a moving obstacle. The direction of movement of the moving obstacle is determined based on the historical brightness change image and the current brightness change image. If the direction of movement of the object is toward the wearer, it is determined that there is a moving obstacle moving toward the wearer in the environment. Otherwise, there is no moving obstacle moving toward the wearer.

[0100] In one example, when it is determined that the area of ​​the moving obstacle in the image is increasing based on the historical brightness change image and the current brightness change image, it is determined that the moving direction of the moving obstacle is toward the wearer.

[0101] S2700: If present, output a second collision warning prompt message.

[0102] In the embodiment of the present application, if there is a moving obstacle, it means that there is a moving obstacle moving towards the wearer, and there is a risk that the moving obstacle will collide with the wearer. In this case, the second collision warning prompt information is output to prompt the wearer to avoid the moving obstacle.

[0103] The second collision warning prompt message can be a prompt message based on at least one of sound, light, text, image, and vibration. The second collision warning prompt message can be different from the first collision warning prompt message. Therefore, the wearer can distinguish between the second collision warning prompt message and the first collision warning prompt message.

[0104] Based on the second collision warning prompt information, the wearer can adjust the direction of movement to avoid being hit by moving obstacles.

[0105] In combination with the above content, it can be seen that in the embodiment of the present application, it is also possible to remind the wearer of the presence of moving obstacles with a tendency to collide.

[0106] In one embodiment of the present application, combined with the above S2100, it can be seen that at least two frames of current brightness change images can also be obtained based on the above S2100. Based on this, the collision warning method provided in the embodiment of the present application further includes the following S2210 before the above S2200:

[0107] S2210: When the current brightness change image consists of at least two frames, fuse the at least two frames of the current brightness change image into one frame of the current brightness change image.

[0108] It is understandable that, when the current brightness change image consists of at least two frames, all current brightness change images corresponding to the current moment can be extracted from the images captured by the event camera by means of time alignment.

[0109] In the embodiment of the present application, a current brightness change image with a larger field of view can be obtained by fusion, so that a more accurate position relationship can be obtained when executing the subsequent S2200.

[0110] In one embodiment of the present application, the above S2200 may be specifically implemented through the following S2220 and S2221:

[0111] S2220: Filter out noise in the current brightness change image.

[0112] In one embodiment of the present application, noise in the current brightness change image may be filtered out by, for example, an integral filtering method, so as to obtain a more accurate current brightness change image.

[0113] S2221. Determine the positional relationship between the handle, the head-mounted device, and obstacles in the environment where the head-mounted device is located based on the current brightness change image after noise is filtered out.

[0114] In an embodiment of the present application, when the current brightness change image is more accurate, a more accurate positional relationship between the handle, the head-mounted device, and obstacles in the environment where the head-mounted device is located can be obtained based on the more accurate current brightness change image.

[0115] The embodiment of the present application further provides a collision warning device 300, which is applied to a head-mounted device. The head-mounted device is equipped with at least one handle, such as Figure 3 As shown, the device 300 includes:

[0116] An acquisition module 310 is configured to acquire a current brightness change image captured by an event camera provided on the handle at a current moment;

[0117] A first determining module 320 is configured to determine a positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located based on the current brightness change image;

[0118] A second determining module 330 is configured to determine, based on the positional relationship, whether the wearer of the head mounted device is at risk of collision;

[0119] The warning module 340 is configured to output a first collision warning prompt message when a collision exists.

[0120] The collision warning device provided in the embodiment of the present application can alert the wearer when there is a risk of collision. In this way, the wearer is prevented from colliding while wearing the head-mounted device.

[0121] In one embodiment of the present application, the second determining module 330 is specifically configured to:

[0122] Determine, based on the current brightness change image, spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point, where the first pixel point is a pixel point corresponding to an obstacle in the environment, and the second pixel point is a pixel point corresponding to the wearer;

[0123] Determining a first distance and a second distance based on the spatial information, the first pixel point, and the second pixel point, where the first distance is a distance between the handle and an obstacle in the environment where the handle is located, and the second distance is a distance between the handle and a wearer of the head mounted device;

[0124] When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, it is determined that the wearer of the head mounted device is at risk of collision.

[0125] In one embodiment of the present application, the acquisition module 310 is further configured to acquire at least one frame of historical brightness change image;

[0126] The second determining module 330 is further configured to determine whether there is a moving obstacle moving toward the wearer in the environment based on the historical brightness change image and the current brightness change image;

[0127] The warning module 340 is further configured to output a second collision warning prompt message when the collision exists.

[0128] In one embodiment of the present application, the collision warning device 300 provided in the embodiment of the present application further includes a fusion module, wherein:

[0129] A fusion module is used to fuse at least two frames of the current brightness change image into one frame of the current brightness change image when the current brightness change image consists of at least two frames.

[0130] In one embodiment of the present application, the first determining module 320 is specifically configured to filter out noise in the current brightness change image;

[0131] The positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located is determined based on the current brightness change image after noise is filtered out.

[0132] The embodiment of the present application further provides a head mounted device 400, the head mounted device comprising any one of the collision warning devices provided in the above device embodiments; or,

[0133] like Figure 4 As shown, it includes a memory 410 and a processor 420, the memory 410 is used to store computer instructions, and the processor 420 is used to call the computer instructions from the memory 410 to execute the collision warning method as described in any one of the above method embodiments.

[0134] An embodiment of the present application further provides a head-mounted system, which includes any head-mounted device 400 provided in the above-mentioned device embodiments and at least one handle, and at least one event camera is provided on any of the handles.

[0135] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the collision warning method according to any one of the above method embodiments.

[0136] The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0137] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0138] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0139] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages, wherein the programming language includes object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or executed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (such as by using an Internet service provider to connect to the Internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.

[0140] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0141] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0142] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0143] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0144] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements to technologies in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A collision warning method, characterized in that: Applied to a head-mounted device, the head-mounted device is adapted to be equipped with at least one handle, and the method includes: Obtaining a current brightness change image captured by the event camera set on the handle at the current moment; determining, based on the current brightness change image, a positional relationship among the handle, the head mounted device, and obstacles in an environment in which the head mounted device is located; determining, based on the positional relationship, whether a wearer of the head mounted device is at risk of collision; If there is a collision, output a first collision warning prompt message; Wherein, determining whether the wearer of the head mounted device is at risk of collision based on the positional relationship includes: Determining, based on the current brightness change image, spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point, where the first pixel point is a pixel point corresponding to an obstacle in the environment, and the second pixel point is a pixel point corresponding to a wearer, and the spatial information of the environment in which the handle is located is the three-dimensional coordinate positions of feature points of objects in the environment relative to the handle, where the objects include at least the wearer, the head-mounted device, the handle, and the obstacle; Determining a first distance and a second distance based on the spatial information, the first pixel point, and the second pixel point, where the first distance is a distance between the handle and an obstacle in the environment where the handle is located, and the second distance is a distance between the handle and a wearer of the head mounted device; When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, it is determined that the wearer of the head mounted device is at risk of collision.

2. The method according to claim 1, characterized in that After acquiring the current brightness change image captured by the event camera provided on the handle at the current moment, the method further includes: Acquire at least one frame of historical brightness change image; determining, based on the historical brightness change image and the current brightness change image, whether there is a moving obstacle in the environment moving toward the wearer; If so, a second collision warning prompt message is output.

3. The method according to claim 1, characterized in that Before determining the positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located based on the current brightness change image, the method further includes: In a case where the current brightness change image consists of at least two frames, the at least two frames of the current brightness change image are fused into one frame of the current brightness change image.

4. The method according to claim 1, wherein The determining, based on the current brightness change image, a positional relationship among the handle, the head mounted device, and obstacles in an environment where the head mounted device is located includes: filtering out noise in the current brightness change image; The positional relationship among the handle, the head mounted device, and obstacles in the environment where the head mounted device is located is determined based on the current brightness change image after noise is filtered out.

5. A collision warning device, characterized in that: Applied to a head-mounted device, the head-mounted device is adapted to be equipped with at least one handle, and the device comprises: An acquisition module, configured to acquire a current brightness change image captured by an event camera provided on the handle at a current moment; a first determining module, configured to determine a positional relationship among the handle, the head mounted device, and obstacles in an environment where the head mounted device is located based on the current brightness change image; a second determining module, configured to determine, based on the positional relationship, whether the wearer of the head mounted device is at risk of collision; A warning module, configured to output a first collision warning prompt message when a collision exists; The second determining module is specifically configured to: Determine, based on the current brightness change image, spatial information of the environment in which the handle is located, a first pixel point, and a second pixel point, where the first pixel point is a pixel point corresponding to an obstacle in the environment, and the second pixel point is a pixel point corresponding to the wearer; Determining a first distance and a second distance based on the spatial information, the first pixel point, and the second pixel point, where the first distance is the distance between the handle and an obstacle in an environment in which the handle is located, and the second distance is the distance between the handle and a wearer of the head-mounted device. The spatial information of the environment in which the handle is located is the three-dimensional coordinate positions of feature points of objects in the environment relative to the handle, where the objects include at least the wearer, the head-mounted device, the handle, and the obstacle. When the difference between the first distance and the second distance and the distance between the wearer and the obstacle are both less than or equal to a preset threshold, it is determined that the wearer of the head mounted device is at risk of collision.

6. A head-mounted device, characterized in that: The head-mounted device comprises the apparatus according to claim 5; or The system comprises a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the collision warning method according to any one of claims 1 to 4.

7. A head-mounted system, characterized in that: The head-mounted system includes the head-mounted device according to claim 6 and at least one handle, and at least one event camera is provided on any of the handles.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the collision warning method according to any one of claims 1 to 4 is implemented.

Images