Road surface information acquisition method and device, augmented reality equipment and storage medium

Abstract

The invention provides a road surface information acquisition method and device, augmented reality equipment and a storage medium. The method comprises the following steps: acquiring depth image information of a target road surface and spectral image information of the target road surface; determining flatness information of the target road surface according to the depth image information; determining light reflection information of the target road surface according to the spectral image information; and determining defect information of the target pavement according to the flatness information and the light reflection information. According to the method, depth information and spectral image information of a target road surface can be acquired firstly; afterwards, according to the depth information and the spectral image information, the place where the road surface is uneven or the light reflection is abnormal can be determined. Finally, the flatness and the light reflection condition can be determined, and it is determined that the pavement has defects. In this way, the user can be informed of the current road surface information, so that the user is helped to avoid a dangerous road surface area.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for acquiring road surface information, an extended reality device, and a storage medium. Background Art

[0002] With the gradual popularization of XR devices (Extended Reality Devices), head-mounted smart mobile extended reality devices similar to AR glasses have become the choice of some people, and are worn and used in medium and low-speed travel scenarios such as cycling, walking, and running.

[0003] However, when wearing an XR device, especially when walking, in order to ensure the safety of the user during walking, a solution is needed to help the user obtain road surface information. Summary of the invention

[0004] The present application provides a road surface information acquisition method, which can effectively acquire road surface defect information.

[0005] In a first aspect, the present application provides a method for obtaining road surface information, the method comprising:

[0006] Acquire depth image information of a target road surface and spectral image information of the target road surface;

[0007] Determining the flatness information of the target road surface according to the depth image information;

[0008] Determining light reflection information of the target road surface according to the spectral image information;

[0009] Defect information of the target road surface is determined according to the flatness information and the light reflection information.

[0010] In some embodiments of the present application, the depth image information includes depth information of each position of the target road surface;

[0011] Determining the flatness information of the target road surface according to the depth image information includes:

[0012] respectively taking the depth information corresponding to the target position in the target road surface as the depth information to be processed, and comparing the depth information to be processed with the reference depth information interval;

[0013] If the depth value corresponding to the depth information to be processed is less than the depth value corresponding to the reference depth information interval, the flatness information indicating that a pit exists at the target position is obtained;

[0014] If the depth value corresponding to the depth information to be processed belongs to the depth value range corresponding to the reference depth information interval, normal flatness information of the target position is obtained;

[0015] If the depth value corresponding to the depth information to be processed is greater than the depth value corresponding to the reference depth information interval, flatness information indicating that a protrusion exists at the target position is obtained.

[0016] In some embodiments of the present application, the spectral image information includes spectral information of each position of the target road surface;

[0017] Determining the light reflection information of the target road surface according to the spectral image information includes:

[0018] Taking the spectral information corresponding to the target position in the target road surface as the spectral information to be processed, and comparing the spectral information to be processed with the reference spectral information interval;

[0019] If the spectral value corresponding to the spectral information to be processed belongs to the spectral value range corresponding to the reference spectral information interval, abnormal light reflection information is obtained at the target position;

[0020] If the spectral information to be processed does not belong to the reference spectral information interval, normal light reflection information of the target position is obtained.

[0021] In some embodiments of the present application, after determining the defect information of the target road surface according to the flatness information and the light reflection information, the method further includes:

[0022] Obtaining relative position information corresponding to each position in the target road surface on the imaging plane;

[0023] Determine the target relative position information corresponding to the defect information on the imaging plane according to the relative position information of each position;

[0024] Imaging is performed according to the target relative position information and the defect information to obtain target image information.

[0025] In some embodiments of the present application, the obtaining of the relative position information corresponding to each position in the target road surface on the imaging plane is collected by a target device;

[0026] The obtaining of relative position information corresponding to each position in the target road surface on the imaging plane includes:

[0027] Acquire posture information of the target device;

[0028] Determining optical axis data information of the target device according to the posture information;

[0029] The relative position information corresponding to each position of the target road surface on the imaging plane is determined according to the optical axis data information.

[0030] In some embodiments of the present application, the acquiring the posture information of the target device includes:

[0031] Acquiring initial posture information of the target device;

[0032] If the angle information corresponding to the initial posture information does not match the reference angle information, it is prompted that the current posture of the target device is abnormal;

[0033] If the angle information corresponding to the initial posture information matches the reference angle information, the initial posture information is determined to be the posture information.

[0034] In some embodiments of the present application, performing imaging according to the target relative position information and the defect information to obtain target image information includes:

[0035] Determining a marking level of the defect information;

[0036] Determining a marking type of the defect information according to the marking level;

[0037] According to the target relative position information corresponding to the imaging plane, a target mark corresponding to the mark type is added to the defect information to obtain target image information including the mark information.

[0038] In a second aspect, the present application further provides a road surface information acquisition device, the device comprising:

[0039] An acquisition module, used to acquire depth image information of a target road surface and spectral image information of the target road surface;

[0040] A processing module, used to determine the flatness information of the target road surface according to the depth image information;

[0041] The processing module is also used to determine the light reflection information of the target road surface according to the spectral image information;

[0042] The processing module is also used to determine defect information of the target road surface according to the flatness information and the light reflection information.

[0043] In a third aspect, the present application also provides an extended reality device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in any one of the road surface information acquisition methods.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps in any one of the road surface information acquisition methods.

[0045] The road surface information acquisition method provided by the present application can first obtain the depth information and spectral image information of the target road surface. Afterwards, the flatness information of the target road surface can be determined based on the depth information, thereby determining the unevenness of the road surface. At the same time, it is also possible to determine whether there are places on the road surface where light reflection is abnormal based on the spectral image information. Finally, if the road surface is uneven or there are places where light reflection is abnormal, it can be determined that there are defects in the road surface. Otherwise, there are no serious defects in the road surface. In this way, the user can be informed of the current road surface information, thereby helping the user avoid dangerous areas on the road surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 is a schematic diagram of a scene of a road surface information acquisition system provided in an embodiment of the present application;

[0048] Figure 2 This is a flow chart of an embodiment of a method for obtaining road surface information in an embodiment of the present application;

[0049] Figure 3 This is a functional module diagram of a road surface information acquisition device in an embodiment of the present application;

[0050] Figure 4 It is a schematic diagram of the structure of the terminal device in the embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0052] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the feature. In the description of the present application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0053] In this application, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. At the same time, it is to be understood that in the specific implementation of this application, when user information, user data and other related data are involved, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data shall comply with relevant laws, regulations and standards of relevant countries and regions.

[0054] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the present application.

[0055] The present application provides a road surface information acquisition method, device, equipment and storage medium, which are described in detail below.

[0056] See also Figure 1 , Figure 1 Schematic diagram of a road surface information acquisition system provided in an embodiment of the present application, the road surface information acquisition system may include an extended reality device 100. Figure 1 The extended reality device 100 in the embodiment may include various types of image sensors. When the extended reality device 100 acquires road surface information through its internal image sensor, the road surface information acquisition method in the present application may be executed.

[0057] In the embodiment of the present application, the extended reality device 100 includes but is not limited to a head-mounted helmet, head-mounted glasses, etc.

[0058] It should be noted that Figure 1The scenario diagram of the road surface information acquisition system shown is only an example. The road surface information acquisition system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the road surface information acquisition system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0059] like Figure 2 As shown, Figure 2 This is a flow chart of an embodiment of a method for obtaining road surface information in an embodiment of the present application. The method for obtaining road surface information may include the following steps 201 to 204:

[0060] 201. Obtain depth image information of a target road surface and spectral image information of the target road surface.

[0061] In the embodiment of the present application, the depth image information of the target road surface can be obtained by a TOF (Time-of-Flight) camera built into the extended reality device. Among them, the TOF camera is a device that determines the distance by measuring the time it takes for light to be emitted from the camera to the object and back. It uses a laser or LED light source to emit light pulses, and then receives the reflected light through a sensor to calculate the distance and depth information of the object. At the same time, spectral image information can be obtained by a spectral camera built into the extended reality device.

[0062] It should be noted that although a variety of image sensors are provided in the extended reality device, the embodiments of the present application do not involve the setting layout of the internal image sensors in the extended reality device. The specific setting layout of the image sensors in the extended reality device can refer to any layout setting, and the embodiments of the present application do not limit it.

[0063] In addition, if you want to obtain the depth image information and spectral image information of the target road surface, you can start the corresponding image sensor to obtain it after the user wears the extended reality device and turns on the device.

[0064] 202. Determine the flatness information of the target road surface according to the depth image information.

[0065] According to the characteristics of TOF cameras, TOF cameras can capture the depth information of the road surface based on the speed of light propagation, and the depth information reflects the different propagation times of light. However, at this time, the device cannot know whether the road surface is flat or uneven. Therefore, a benchmark flatness threshold can be set, and the depth information of each position on the road surface can be compared with the flatness threshold. If the depth information of a certain position exceeds the threshold or is lower than the threshold, it is considered uneven. In this way, the flatness information of each position on the road surface can be obtained, and the user can determine whether the road surface is flat based on the flatness information, thereby deciding how to be careful when walking.

[0066] 203. Determine light reflection information of the target road surface according to the spectral image information.

[0067] In addition to the unevenness of the road surface, there are other types of defects, such as puddles on the road surface. In this case, the road surface will become slippery, thereby increasing the probability of people falling when walking on the road surface. Therefore, in an embodiment of the present application, after obtaining the spectral image information of each position of the road surface, the spectral reflection of the road surface can be analyzed to see whether its reflection is similar to the reflection of liquids such as water surface and oil stains. For example: if the spectral reflection characteristics of a certain place on the road surface are similar to the spectral reflection characteristics of liquid, it can be determined that there is liquid on the road surface at that place. In this way, it can also help users avoid areas of the road surface that are easy to slip.

[0068] 204. Determine defect information of the target road surface according to the flatness information and the light reflection information.

[0069] Finally, by combining the flatness information and light reflection information in the above steps, a highly comprehensive set of road surface defect information can be obtained, which can help users comprehensively judge the current road conditions. For example, users can be prompted by voice or other means that the current road conditions are relatively complicated, and a prompt voice with relevant semantics such as caution can be issued.

[0070] The road surface information acquisition method provided by the present application can first obtain the depth information and spectral image information of the target road surface. Afterwards, the flatness information of the target road surface can be determined based on the depth information, thereby determining the unevenness of the road surface. At the same time, it is also possible to determine whether there are places on the road surface where light reflection is abnormal based on the spectral image information. Finally, if the road surface is uneven or there are places where light reflection is abnormal, it can be determined that there are defects in the road surface. Otherwise, there are no serious defects in the road surface. In this way, the user can be informed of the current road surface information, thereby helping the user avoid dangerous areas on the road surface.

[0071] In order to better implement the embodiment of the present application, in one embodiment of the present application, the depth image information includes depth information of each position of the target road surface; and according to the depth image information, determining the flatness information of the target road surface includes:

[0072] The depth information corresponding to the target position in the target road surface is respectively used as the depth information to be processed, and the depth information to be processed is compared with the reference depth information interval; if the depth value corresponding to the depth information to be processed is smaller than the depth value corresponding to the reference depth information interval, the flatness information that there is a pit at the target position is obtained; if the depth value corresponding to the depth information to be processed belongs to the depth value range corresponding to the reference depth information interval, the normal flatness information of the target position is obtained; if the depth value corresponding to the depth information to be processed is larger than the depth value corresponding to the reference depth information interval, the flatness information that there is a protrusion at the target position is obtained.

[0073] The above embodiment provides a solution for determining whether a road surface is flat according to a flatness threshold. However, in order to ensure that more detailed flatness information can be obtained, the embodiment of the present application also provides a solution as described above.

[0074] Among them, the flatness threshold can be set as a reference depth information interval. When the depth information to be processed is less than the minimum depth information in the reference depth information interval, it can be determined that the corresponding target position is a road pothole. On the contrary, if the depth information to be processed is greater than the maximum depth information in the reference depth information interval, it can be proved that there is a road bump at the corresponding target position, and both road bumps and potholes may increase the danger level of the user's walking. In addition, if the depth information to be processed is in the reference depth information interval, it proves that there are no potholes or bumps at the target position, and it belongs to a flat road section.

[0075] In order to better implement the embodiment of the present application, in one embodiment of the present application, the spectral image information includes spectral information of each position of the target road surface; according to the spectral image information, determining the light reflection information of the target road surface includes:

[0076] The spectral information corresponding to the target position in the target road surface is taken as the spectral information to be processed, and the spectral information to be processed is compared with the reference spectral information interval; if the spectral value corresponding to the spectral information to be processed belongs to the spectral value range corresponding to the reference spectral information interval, abnormal light reflection information of the target position is obtained; if the spectral information to be processed does not belong to the reference spectral information interval, normal light reflection information of the target position is obtained.

[0077] The above embodiment provides a method of comparing the spectral information at the corresponding position of the road surface with the spectral information of the set liquid, so as to determine whether the two are similar, and then determine whether there is liquid at the target position corresponding to the road surface. However, in addition to liquid, it also includes conditions such as road icing. Therefore, in the embodiment of the present application, different reference spectral intervals can also be set, and each reference spectral interval corresponds to a road surface abnormality, such as liquid, oil, ice, etc. If the spectral information of the corresponding road surface position is within any reference spectral interval, it can be determined that the corresponding abnormality exists at the corresponding position of the target road surface.

[0078] In order to better implement the embodiment of the present application, in one embodiment of the present application, after determining the defect information of the target road surface according to the flatness information and the light reflection information, the method further includes:

[0079] Obtain the relative position information corresponding to each position in the target road surface on the imaging plane; determine the target relative position information corresponding to the defect information on the imaging plane based on the relative position information of each position; perform imaging based on the target relative position information and the defect information to obtain the target image information.

[0080] The above embodiment provides an implementation method for prompting the user of abnormalities on the road surface through voice. However, according to the above embodiment, the solution provided by the present application can be used for an extended reality device. In addition to being equipped with a corresponding image sensor, the extended reality device is also equipped with a corresponding display device. Therefore, in the embodiment of the present application, the location where there is a defect on the road surface can be imaged on the display device, so that the user can visually identify the specific defect location.

[0081] In the embodiment of the present application, imaging can be performed by any imaging method, such as 2D imaging, etc., which is not limited in the embodiment of the present application. In addition, 2D imaging can adopt any 2D imaging technology, which is also not limited in the embodiment of the present application.

[0082] In order to better implement the embodiments of the present application, in one embodiment of the present application, the relative position information corresponding to each position in the target road surface on the imaging plane is acquired by the target device; the relative position information corresponding to each position in the target road surface on the imaging plane is acquired, including:

[0083] Acquire the posture information of the target device; determine the optical axis data information of the target device according to the posture information; and determine the relative position information corresponding to each position of the target road surface on the imaging plane according to the optical axis data information.

[0084] According to the above embodiments, the imaging of the present application can be a 2D imaging method. In order to achieve the imaging effect, the present application can also use 3D imaging to image the defects of the road surface. However, if 3D imaging is required, the position of the defect in the road surface needs to correspond to the imaging plane. Therefore, in order to solve this problem, the present application provides a position corresponding solution.

[0085] Specifically, in the embodiment of the present application, since the image information of the road surface is obtained by an extended reality device, the target device is the extended reality device. In addition, the extended reality device may also include a gyroscope, and the posture information of the extended reality device is determined by the gyroscope.

[0086] Among them, in the embodiment of the present application, the optical axis data can be understood as the shooting direction of the image sensor. Assuming that the user wears the extended reality device in a conventional posture of looking straight ahead, the direction of the optical axis data is parallel to the ground. In this case, it can be understood that the angle between the road surface and the plane where the display screen in the extended reality device is located is 90 degrees. At this time, the road surface information that can be captured by the field of view in the extended reality device can be mapped to the display screen according to a 90-degree mapping method.

[0087] Of course, it should be noted that when the user looks straight ahead, the direction of the optical axis data is not necessarily parallel to the ground. For example: if the shooting direction of the image sensor is appropriately downward, the optical axis data is not parallel to the ground, and there is a certain angle θ. In this case, it is considered that the current mapping angle between the ground and the display screen is the angle θ, and the 3D image can be imaged according to the mapping angle. At the same time, in the embodiment of the present application, the 3D imaging method can be any 3D imaging method, which is not limited in the embodiment of the present application.

[0088] In order to better implement the embodiment of the present application, in one embodiment of the present application, obtaining the posture information of the target device includes:

[0089] The initial posture information of the target device is obtained; if the angle information corresponding to the initial posture information does not match the reference angle information, it is prompted that the current posture of the target device is abnormal; if the angle information corresponding to the initial posture information matches the reference angle information, the initial posture information is determined to be the posture information.

[0090] Under normal circumstances, when a user walks forward, the user's line of sight is also forward. However, in some cases, when a user walks forward, the user's head will tilt to the side, causing the line of sight to leave the target road surface. At this time, since the user is still on the target road surface and walking forward, there is no need to image other objects seen by the user. Therefore, when obtaining the initial posture information of the target device, it is also necessary to determine whether the user's line of sight is the same as the forward direction.

[0091] Specifically, after the user wears it normally, when the head is in a normal state without pitch or twist, the corresponding system device is in the default posture and corresponds to the default system parameters. In the default posture, the image sensor captures the ground scene in the forward direction, the optical axis data of the sensor module forms an angle θ with the horizontal ground, and the horizontal plane projection of the optical axis is consistent with the forward direction. It should be noted that the field of view angle of the image sensor is α, and the default height H of the module in the default state is set by the user according to his own situation. The corresponding default maximum detection range is an isosceles trapezoidal area with a height of LR in the forward direction, and a visual blind area with a length range of L0 is generated. The gyroscope in the target device is fixed relative to the image sensor to detect the posture changes of the device as the user's head moves, and then correct the sampling orientation parameters of the image sensor. When the user's head rotates and pitches, the height and optical axis direction of the image sensor change accordingly. The height change is H', which can be estimated by the default height H and pitch situation; the angle between the optical axis and the horizontal plane changes to θ', and the horizontal projection forms an angle δ with the forward direction, which can be calculated based on the inertial strategy module data. The corresponding maximum detection range is an isosceles trapezoidal area with a height of L'R at an angle of δ to the forward direction, and a visual blind area with a length range of L'0 is generated. If the user's head rotation and pitch angle are set to an allowable range to ensure the validity of the detection results, when the head movement exceeds the allowable amplitude range, the corresponding visual data will not be processed and the user will be prompted.

[0092] In order to better implement the embodiments of the present application, in one embodiment of the present application, imaging is performed according to the target relative position information and the defect information to obtain the target image information, including:

[0093] Determine the marking level of the defect information; determine the marking type of the defect information according to the marking level; add a target mark corresponding to the marking type to the defect information based on the target relative position information on the imaging plane to obtain target image information including the marking information.

[0094] In order to further improve the recognition of defects, the defect information can be marked when imaging the defect information, such as by highlighting, so that users can notice the defects more easily.

[0095] Different levels can be set for different defect types. For example, depression is one level, protrusion is one level, and slippery liquid on the surface is one level. At the same time, different highlight marks can be set for different levels. In this way, when performing 3D imaging of defect information, different levels of highlight marks can be set for different defect types in the 3D picture, so that users can distinguish different defect types more intuitively.

[0096] In order to better implement the road surface information acquisition method in the embodiment of the present application, in addition to the road surface information acquisition method, the embodiment of the present application also provides a road surface information acquisition device, such as Figure 3 As shown, the device 300 includes:

[0097] An acquisition module 301 is used to acquire depth image information of a target road surface and spectral image information of the target road surface;

[0098] The processing module 302 is used to determine the flatness information of the target road surface according to the depth image information;

[0099] The processing module 302 is also used to determine the light reflection information of the target road surface according to the spectral image information;

[0100] The processing module 302 is further used to determine defect information of the target road surface according to the flatness information and the light reflection information.

[0101] The road surface information acquisition device provided by the present application can first acquire the depth information and spectral image information of the target road surface through the acquisition module 301. Afterwards, the processing module 302 can determine the flatness information of the target road surface based on the depth information, thereby determining the unevenness of the road surface. At the same time, it can also be determined whether there are places where the road surface has abnormal light reflection based on the spectral image information. Finally, if the road surface is uneven or there are places where light reflection is abnormal, it can be determined that there are defects in the road surface. Otherwise, there are no serious defects in the road surface. In this way, the user can be informed of the current road surface information, thereby helping the user avoid dangerous areas on the road surface.

[0102] In some embodiments of the present application, the depth image information includes depth information of each position of the target road surface, and the processing module 302 is specifically used to:

[0103] The depth information corresponding to the target position in the target road surface is respectively used as the depth information to be processed, and the depth information to be processed is compared with the reference depth information interval;

[0104] If the depth value corresponding to the depth information to be processed is less than the depth value corresponding to the reference depth information interval, the flatness information of the pit existing at the target position is obtained;

[0105] If the depth value corresponding to the depth information to be processed falls within the depth value range corresponding to the reference depth information interval, normal flatness information of the target position is obtained;

[0106] If the depth value corresponding to the depth information to be processed is greater than the depth value corresponding to the reference depth information interval, flatness information indicating that a protrusion exists at the target position is obtained.

[0107] In some embodiments of the present application, the spectral image information includes spectral information of each position of the target road surface, and the processing module 302 is specifically used to:

[0108] The spectral information corresponding to the target position in the target road surface is respectively used as the spectral information to be processed, and the spectral information to be processed is compared with the reference spectral information interval;

[0109] If the spectral value corresponding to the spectral information to be processed belongs to the spectral value range corresponding to the reference spectral information interval, abnormal light reflection information is obtained at the target position;

[0110] If the spectral information to be processed does not belong to the reference spectral information interval, normal light reflection information of the target position is obtained.

[0111] In some embodiments of the present application, the processing module 302 is further configured to:

[0112] Obtaining the relative position information corresponding to each position in the target road surface on the imaging plane;

[0113] According to the relative position information of each position, determine the target relative position information corresponding to the defect information on the imaging plane;

[0114] Imaging is performed based on the target relative position information and defect information to obtain target image information.

[0115] In some embodiments of the present application, the relative position information corresponding to each position in the target road surface on the imaging plane is acquired by the target device, and the processing module 302 is specifically used to:

[0116] Get the posture information of the target device;

[0117] Determine the optical axis data information of the target device according to the posture information;

[0118] According to the optical axis data information, the relative position information corresponding to each position of the target road surface on the imaging plane is determined.

[0119] In some embodiments of the present application, the processing module 302 is further configured to:

[0120] Get the initial posture information of the target device;

[0121] If the angle information corresponding to the initial posture information does not match the reference angle information, it is prompted that the current posture of the target device is abnormal;

[0122] If the angle information corresponding to the initial posture information matches the reference angle information, the initial posture information is determined to be the posture information.

[0123] In some embodiments of the present application, the processing module 302 is further configured to:

[0124] Determine the marking level of defect information;

[0125] According to the marking level, determine the marking type of the defect information;

[0126] Corresponding to the target relative position information on the imaging plane, a target mark corresponding to the mark type is added to the defect information to obtain target image information including the mark information.

[0127] The embodiment of the present application also provides an extended reality device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in any of the road surface information acquisition methods in the embodiments of the present application. The extended reality device integrates any of the road surface information acquisition methods provided in the embodiments of the present application, such as Figure 4 As shown, it shows a schematic diagram of the structure of the extended reality device involved in the embodiment of the present application, specifically:

[0128] The extended reality device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 4 The structure of the extended reality device shown in the figure does not constitute a limitation on the extended reality device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0129] The processor 401 is the control center of the extended reality device, and uses various interfaces and lines to connect various parts of the entire extended reality device. By running or executing the software programs and / or modules stored in the memory 402, and calling the data stored in the memory 402, the processor 401 performs various functions of the extended reality device and processes data, thereby monitoring the extended reality device as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401 .

[0130] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the extended reality device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0131] The extended reality device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0132] The extended reality device may further include an input unit 404, which may be used to receive input digital or character information, and generate microphone, joystick, optical or trackball signal input related to user settings and function control.

[0133] Although not shown, the extended reality device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the extended reality device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, such as:

[0134] Acquire depth image information of the target road surface and spectral image information of the target road surface;

[0135] Determine the flatness information of the target road surface according to the depth image information;

[0136] Determine the light reflection information of the target road surface according to the spectral image information;

[0137] Determine the defect information of the target road surface based on the flatness information and light reflection information.

[0138] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0139] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the road surface information acquisition methods provided in the embodiment of the present application. For example, the computer program loaded by the processor can execute the following steps:

[0140] Acquire depth image information of the target road surface and spectral image information of the target road surface;

[0141] Determine the flatness information of the target road surface according to the depth image information;

[0142] Determine the light reflection information of the target road surface according to the spectral image information;

[0143] Determine the defect information of the target road surface based on the flatness information and light reflection information.

[0144] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0145] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0146] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0147] The above is a detailed introduction to a road surface information acquisition method and device provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for obtaining road surface information, characterized in that: The method comprises: Acquire depth image information of a target road surface and spectral image information of the target road surface; Determining the flatness information of the target road surface according to the depth image information; Determining light reflection information of the target road surface according to the spectral image information; Defect information of the target road surface is determined according to the flatness information and the light reflection information.

2. The method for obtaining road surface information according to claim 1, characterized in that: The depth image information includes depth information of each position of the target road surface; Determining the flatness information of the target road surface according to the depth image information includes: respectively taking the depth information corresponding to the target position in the target road surface as the depth information to be processed, and comparing the depth information to be processed with the reference depth information interval; If the depth value corresponding to the depth information to be processed is less than the depth value corresponding to the reference depth information interval, the flatness information indicating that a pit exists at the target position is obtained; If the depth value corresponding to the depth information to be processed belongs to the depth value range corresponding to the reference depth information interval, normal flatness information of the target position is obtained; If the depth value corresponding to the depth information to be processed is greater than the depth value corresponding to the reference depth information interval, flatness information indicating that a protrusion exists at the target position is obtained.

3. The method for obtaining road surface information according to claim 1, characterized in that: The spectral image information includes spectral information of each position of the target road surface; Determining the light reflection information of the target road surface according to the spectral image information includes: Taking the spectral information corresponding to the target position in the target road surface as the spectral information to be processed, and comparing the spectral information to be processed with the reference spectral information interval; If the spectral value corresponding to the spectral information to be processed belongs to the spectral value range corresponding to the reference spectral information interval, abnormal light reflection information is obtained at the target position; If the spectral information to be processed does not belong to the reference spectral information interval, normal light reflection information of the target position is obtained.

4. The method for obtaining road surface information according to claim 1, characterized in that: After determining the defect information of the target road surface according to the flatness information and the light reflection information, the method further includes: Obtaining relative position information corresponding to each position in the target road surface on the imaging plane; Determine the target relative position information corresponding to the defect information on the imaging plane according to the relative position information of each position; Imaging is performed according to the target relative position information and the defect information to obtain target image information.

5. The method for obtaining road surface information according to claim 4, characterized in that: The acquisition of the relative position information corresponding to each position in the target road surface on the imaging plane is collected by a target device; The obtaining of relative position information corresponding to each position in the target road surface on the imaging plane includes: Acquire posture information of the target device; Determining optical axis data information of the target device according to the posture information; The relative position information corresponding to each position of the target road surface on the imaging plane is determined according to the optical axis data information.

6. The method for obtaining road surface information according to claim 5, characterized in that: The acquiring the posture information of the target device includes: Acquiring initial posture information of the target device; If the angle information corresponding to the initial posture information does not match the reference angle information, it is prompted that the current posture of the target device is abnormal; If the angle information corresponding to the initial posture information matches the reference angle information, the initial posture information is determined to be the posture information.

7. The method for obtaining road surface information according to claim 4, characterized in that: The performing imaging according to the target relative position information and the defect information to obtain target image information includes: Determining a marking level of the defect information; Determining a marking type of the defect information according to the marking level; According to the target relative position information corresponding to the imaging plane, a target mark corresponding to the mark type is added to the defect information to obtain target image information including the mark information.

8. A road surface information acquisition device, characterized in that: The device comprises: An acquisition module, used to acquire depth image information of a target road surface and spectral image information of the target road surface; A processing module, used to determine the flatness information of the target road surface according to the depth image information; The processing module is also used to determine the light reflection information of the target road surface according to the spectral image information; The processing module is also used to determine defect information of the target road surface according to the flatness information and the light reflection information.

9. An extended reality device, characterized in that: The extended reality device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in the road surface information acquisition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the road surface information acquisition method according to any one of claims 1 to 7.

Images