VR display method, device, electronic device and storage medium for road damage

Abstract

The embodiment of the present invention provides a VR display method for road damage, which obtains the user's line of sight data for a reference display interface, and obtains the UI component of the reference display interface and the display data corresponding to the UI component; determines the VR display data corresponding to the reference display interface based on the line of sight data, the UI component and the display data; and displays the VR display data through a VR device. The VR display data corresponding to the reference display interface is determined by the user's line of sight data, the UI component of the reference display interface and the display data corresponding to the UI component, and the VR display data is displayed to the user through the VR device, so that the user can see a three-dimensional road damage interface through the VR device. Since the VR display data is determined in combination with the line of sight data, it can be more in line with the user's line of sight habits, thereby improving the browsing efficiency of road damage.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a VR display method, device, electronic equipment and storage medium for road hazards. Background Art

[0002] As the mileage of highways increases, more and more car owners will choose highways for travel. The maintenance of highways is becoming more and more important. It is necessary to timely discover the road diseases of highways and repair and maintain the discovered road diseases. Due to the large amount of disease data, all road disease data are displayed in the reference display interface during the road disease display process. Generally speaking, the reference display interface of road diseases is flat, the road diseases are not intuitive enough, and it takes a lot of time to read the reference display interface. Therefore, the existing road disease display has the problem of low road disease browsing efficiency. Summary of the invention

[0003] The embodiment of the present invention provides a VR display method for road damage, aiming to solve the problem of low browsing efficiency of road damage in the existing display of road damage. VR display data corresponding to the reference display interface is determined by the user's line of sight data, the UI component of the reference display interface, and the display data corresponding to the UI component, and the VR display data is displayed to the user through the VR device, so that the user can see a three-dimensional road damage interface through the VR device. Since the VR display data is determined in combination with the line of sight data, it can be more in line with the user's line of sight habits, thereby improving the browsing efficiency of road damage.

[0004] In a first aspect, an embodiment of the present invention provides a method for VR display of road hazards, the method comprising:

[0005] Acquire user's sight line data for a reference display interface, and acquire UI components of the reference display interface and display data corresponding to the UI components;

[0006] Determine VR display data corresponding to the reference display interface based on the line of sight data, the UI component, and the display data;

[0007] VR display data is displayed through VR devices.

[0008] Optionally, obtaining the user's sight line data for the reference display interface includes:

[0009] When the user browses the reference display interface, adjusting the clarity of the reference display interface;

[0010] In the process of adjusting the definition of the reference display interface, obtaining a facial image sequence of the user;

[0011] The sight line data of the user with respect to the reference display interface is determined based on the facial image sequence.

[0012] Optionally, determining VR display data corresponding to the reference display interface based on the line of sight data, the UI component, and the display data includes:

[0013] Performing three-dimensional processing on the UI component according to the sight line data to obtain a three-dimensional UI component corresponding to the UI component, wherein each UI component corresponds to one three-dimensional UI component;

[0014] VR display data corresponding to the reference display interface is determined based on the three-dimensional UI component and the display data.

[0015] Optionally, performing three-dimensional processing on the UI component according to the line of sight data to obtain a three-dimensional UI component corresponding to the UI component includes:

[0016] Determining the sight line weight of the user for each of the UI components according to the sight line data;

[0017] Determining three-dimensional shape parameters of the UI components based on the shapes of the respective UI components;

[0018] Determining three-dimensional dynamic parameters of each of the UI components according to the sight line weight;

[0019] The three-dimensional UI component corresponding to each of the UI components is determined based on the three-dimensional shape parameters and the three-dimensional dynamic parameters.

[0020] Optionally, the sight line data determines the sight line weight of the user for each UI component, including:

[0021] Determine the number of times the user has looked at each of the UI components and the duration of their looking according to the sight data;

[0022] The gaze weight of the user for each of the UI components is determined based on the number of gaze times and the duration of the gaze stay.

[0023] Optionally, the determining VR display data corresponding to the reference display interface based on the three-dimensional UI component and the display data includes:

[0024] Classifying the display data to determine target display data that needs to be three-dimensionally processed;

[0025] Performing three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component;

[0026] Based on the three-dimensional UI component and the corresponding three-dimensional target data, VR display data is determined.

[0027] Optionally, performing three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component includes:

[0028] Determine a first three-dimensional shape parameter and a first three-dimensional dynamic parameter of the three-dimensional UI component;

[0029] Determine a second three-dimensional shape parameter and a second three-dimensional dynamic parameter corresponding to the target display data based on the first three-dimensional shape parameter and the first three-dimensional dynamic parameter;

[0030] The three-dimensional target data corresponding to the three-dimensional UI component is determined based on the second three-dimensional shape parameter and the second three-dimensional dynamic parameter.

[0031] In a second aspect, an embodiment of the present invention further provides a VR display device for road damage, wherein the VR display device for road damage comprises:

[0032] An acquisition module, used to acquire the user's sight line data for a reference display interface, and acquire the UI components of the reference display interface and the display data corresponding to the UI components;

[0033] A determination module, configured to determine VR display data corresponding to the reference display interface based on the sight line data, the UI component, and the display data;

[0034] The display module is used to display VR display data through VR devices.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the VR display method for road damage provided in an embodiment of the present invention when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the VR display method for road damage provided in the embodiment of the invention are implemented.

[0037] In an embodiment of the present invention, the user's line of sight data for a reference display interface is obtained, as well as the UI component of the reference display interface and the display data corresponding to the UI component; VR display data corresponding to the reference display interface is determined based on the line of sight data, the UI component and the display data; and the VR display data is displayed through a VR device. The VR display data corresponding to the reference display interface is determined based on the user's line of sight data, the UI component of the reference display interface and the display data corresponding to the UI component, and the VR display data is displayed to the user through the VR device, so that the user can see a three-dimensional road damage interface through the VR device. Since the VR display data is determined in combination with the line of sight data, it can be more in line with the user's line of sight habits, thereby improving the browsing efficiency of road damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 It is a flow chart of a VR display method of road damage provided by an embodiment of the present invention;

[0040] Figure 2 is a structural schematic diagram of a VR display device for road damage provided in an embodiment of the present invention;

[0041] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, Figure 1 The method flow chart of a method for VR display of road damage provided by an embodiment of the present invention. The VR display method for road damage comprises the following steps:

[0044] 101. Obtain user's sight line data for a reference display interface, and obtain UI components of the reference display interface and display data corresponding to the UI components.

[0045] In an embodiment of the present invention, the VR display method of the road damage can be deployed in a VR display system of the road damage. The VR display system of the road damage can include a server, a road damage display terminal, a VR device and a face image acquisition device, wherein the server is respectively connected to the road damage display terminal, the VR device and the face image acquisition device for communication. The reference display interface can be displayed through the road damage display terminal. The server stores road information and maps. The road information can include road signs, road parameters, road damage data, etc. The road damage data includes the degree of road damage, the type of road damage, past damage repair data and technical solutions, etc. Specifically, each road damage in the road damage data can store data such as specific types, past repair records and damage identification. The road damage display terminal can be an electronic device for displaying data content, such as a smart display screen, a smart phone, a computer, a tablet, etc. with a data display function.

[0046] The server can be connected to the road damage display terminal, VR device and face image acquisition device by wired communication or wireless communication. In the embodiment of the present invention, wireless communication is preferred, for example, the server can be connected to the road damage display terminal, VR device and face image acquisition device respectively by wireless communication such as 4G / 5G network, wifi wireless fidelity network, Bluetooth, etc.

[0047] The above-mentioned road signs may be various signs, markings and signal facilities set up on the road to indicate the direction of vehicle travel, speed limit, traffic prohibition regulations and other information, such as traffic lights, road signs, markings, roadblocks, etc. The above-mentioned road parameters refer to various quantitative indicators that describe road characteristics and performance, and usually may include information such as the number of lanes, lane width and road width.

[0048] The reference display interface can be a display interface of a road damage display terminal. The reference display interface has UI components and display data corresponding to the UI components. The reference display interface is a two-dimensional display interface. The interface for displaying road damage data, specifically, the reference display interface can display the road damage data required by the user. The reference display interface can add UI components and display data of corresponding UI components. The UI components can be substantial functional components such as windows, button components, input boxes, drop-down boxes, road damage lists, etc. on the reference display interface. The display data of the UI components can be digital display data corresponding to the UI components, such as map data of road damage, number of road damage, event display data, standard time data, personnel image data, etc. It should be noted that the reference display interface and UI components need to be designed according to the actual plan, and the display data of the corresponding UI components should be designed according to the designed UI components, and diversified design can be achieved through combination.

[0049] The UI components of the reference display interface and the display data corresponding to the UI components can be obtained by referring to the underlying data of the display interface. Specifically, the underlying data includes the anchor position of the UI component, and the anchor position of the UI component refers to the position of the UI component relative to the parent container. The RectTransform component in Unity can be used to determine the anchor position of the UI component. The display data corresponding to the UI component can be determined in the server through the uniform resource locator corresponding to the UI component.

[0050] The above sight data may include the user's sight data for each UI component, and the above sight data may indicate the angle and intensity of the user's attention, and the degree of the user's attention to each UI component may be determined by the angle and intensity of the user's attention. Specifically, the UI component that the user is paying attention to may be determined in the reference display interface according to the user's attention angle, and the degree of the user's attention to the UI component may be determined by the intensity of the user's attention to the UI component.

[0051] The above sight data can be determined based on information such as the user's eyesight, focus, and pupil contraction degree. The reference display interface can be adjusted based on the above sight data. Specifically, a facial image sequence of the user looking at the reference display interface can be obtained through a facial image acquisition device, and the user's sight data can be extracted based on the facial image sequence. The user's sight data can be extracted from the facial image sequence through algorithms such as Optical Flow, Active Appearance Models, DEAM, and ACF.

[0052] Specifically, before the user uses the VR device, the user can watch the reference display interface first. When the user watches the reference display interface, the face image of the user is collected through the face image device to obtain a face image sequence. The face image device uploads the collected face image sequence to the server, and the server performs image processing on the face image sequence to extract the user's line of sight data from the face image sequence. The server can perform three-dimensional face construction through the face image sequence, and perform three-dimensional construction of the display interface through the road disease display terminal, and then construct a three-dimensional mapping model between the user's face and the reference display interface to determine the user's line of sight data for the reference display interface.

[0053] 102. Determine VR display data corresponding to the reference display interface based on the line of sight data, UI components, and display data.

[0054] In an embodiment of the present invention, after obtaining the line of sight data, UI components and display data, the server can analyze and process the above line of sight data, UI components and display data of UI components to obtain VR display data corresponding to the reference display interface.

[0055] During the process of analyzing and processing the above-mentioned sight line data by the server, the sight line habit characteristics of the user are determined based on the capture of the user's eyes by each UI component in the sight line data and the length of time the eyes are focused on a certain point. The above-mentioned sight line habit characteristics can be which part of the screen the user tends to focus on when looking at the screen of the road defect display terminal, and which parts the focus is often placed on when using the screen to check for defects. By analyzing the user's sight line habit characteristics, the UI components on the reference display interface can be adaptively laid out, so that after the UI components are three-dimensionalized, they will not hinder the user's sight of the focused part, and the UI components that the user focuses on can be laid out in a position that conforms to the user's sight line habits, thereby improving the user's VR experience.

[0056] Specifically, the server obtains the eye image sequence of the user in the reference display interface within a period of time through the face image acquisition device, and processes the above eye image sequence through the eye algorithm. Specifically, the above eye image sequence can be processed through the eye algorithm such as OpticalFlow, Active Appearance Models, DEAM, ACF, etc. to obtain the eye focus point. The above eye focus point can be the basis for the human eye to clearly observe the objects around the point, and the above eye focus point is dynamically captured. According to the capture result, the images with the eye focus point falling on the same part of the reference display interface are classified into different types of image sequences, and the duration of each type of image sequence is calculated. According to the length of each type of image sequence, a small number of UI components can be laid out for the part with a shorter eye focus stay time or the above-mentioned fading processing can be performed on this part of the UI components, and a large number of UI components can be laid out for the part of the reference display interface with a longer eye focus stay time or the above-mentioned border enhancement processing can be performed on this part of the UI components, and the VR display data corresponding to the reference display interface can be obtained and displayed to the user through the head display device, so that a reasonable UI component layout can be performed according to the user's visual habits.

[0057] In the process of processing the above data, the display data corresponding to the above UI components can be processed through the two-dimensional display conversion stereoscopic display technology. Specifically, the original data processed by the above server can be converted into three-dimensional display data through two-dimensional display conversion stereoscopic display technologies such as Parallax Mapping, Disparity Map, 3D display, and 3D modeling. The above original data includes the above map data, road signs, road parameters, road disease data, etc. Specifically, by processing the original data in three dimensions, the above three-dimensional processing can perform 3D modeling on the UI components that need to be three-dimensionalized through the above two-dimensional display conversion three-dimensional display technology to obtain three-dimensional UI components, and the corresponding three-dimensionally processed original data is mapped on the corresponding three-dimensional UI components, and the corresponding mapped three-dimensional UI components are used as VR display data.

[0058] 103. Display VR display data through VR devices.

[0059] In an embodiment of the present invention, after obtaining the VR display data, the server may transmit the VR display data to the VR device. After the VR device receives and decodes the VR display data, it displays the VR display data to the user of the VR device.

[0060] Specifically, during the display process, the VR device also includes an interface controller. The user can use the above controller to close the VR display data that does not need to be displayed, and can also observe the VR display data 360° and arrange the VR display data in three dimensions through the controller.

[0061] In a possible embodiment, the user's line of sight data is monitored in real time through the built-in line of sight capture device of the VR device, so as to achieve the purpose of operating the three-dimensional UI component in the reference display interface without the above-mentioned controller. Specifically, by calculating the length of time the user's eyes stay on the above-mentioned three-dimensional UI component and comparing it with the preset length of time, it is determined whether the user needs to operate the above-mentioned three-dimensional UI component. More specifically, through the above-mentioned line of sight data extraction operation, the length of time the user's eyes stay on the close button on the three-dimensional UI component is calculated, and a judgment is made based on the preset stay time threshold. If the stay time is less than the length threshold, the three-dimensional UI component is closed.

[0062] In an embodiment of the present invention, the user's line of sight data for a reference display interface is obtained, as well as the UI components of the reference display interface and the display data corresponding to the UI components; VR display data corresponding to the reference display interface is determined based on the line of sight data, the UI components and the display data; and the VR display data is displayed through a VR device. The VR display data corresponding to the reference display interface is determined by the user's line of sight data, the UI components of the reference display interface and the display data corresponding to the UI components, and the VR display data is displayed to the user through the VR device, so that the user can see a three-dimensional road damage interface through the VR device. Since the VR display data is determined in combination with the line of sight data, it can be more in line with the user's line of sight habits, thereby improving the browsing efficiency of road damage.

[0063] Optionally, when obtaining the user's line of sight data for a reference display interface, it also includes adjusting the clarity of the reference display interface when the user browses the reference display interface. During the process of adjusting the clarity of the reference display interface, a facial image sequence of the user is obtained, and the user's line of sight data for the reference display interface is determined based on the facial image sequence.

[0064] In the embodiment of the present invention, the human eye can see clearly the focused objects, but the objects out of focus will be in a blurred display state. According to this feature, in the process of the clarity change of the above-mentioned reference display interface, the user's more accurate sight habits can be obtained. For example, when the overall clarity of the reference display interface changes, the user will actively look for an area that is more in line with the sight habits for viewing. Therefore, by obtaining the user's face image sequence in the process of adjusting the clarity of the reference display interface, and determining the user's sight data for the reference display interface based on the face image sequence, sight data that is more in line with the user's habits can be obtained.

[0065] The clarity of the above-mentioned reference display interface can be controlled on-site by the road defect display terminal through the display screen resolution, or it can be controlled by the data resolution of the reference display interface sent by the server. The more pixels displayed, the higher the resolution. Higher resolution means more details and clearer images.

[0066] In the clarity adjustment process, specifically, a facial image acquisition device can be used to obtain a sequence of eye images of the user in a reference display interface over a period of time, and eye algorithms such as Optical Flow, Active Appearance Models, DEAM, and ACF can be used to dynamically capture the eye focus points in the above eye image sequence. When the user is relaxed, the eye focus point stays at a certain position of the disease display in the reference display interface, and the UI components in the reference display interface are adjusted to the corresponding position. The above corresponding position is the eye focus point position of the human eye when relaxed, and it is also the position where the human eye can see things clearly. Specifically, it is necessary to enable the user to clearly obtain the data corresponding to the disease when viewing the disease.

[0067] More specifically, by dynamically capturing the position of the eye focus point, dynamic sight line data can be obtained. Based on the dynamic sight line data, the user's sight line habits can be located and captured more quickly, thereby improving the user's VR experience.

[0068] Optionally, in the step of determining the VR display data corresponding to the reference display interface based on the line of sight data, UI components and display data, the UI components can also be three-dimensionalized according to the line of sight data to obtain three-dimensional UI components corresponding to the UI components, each UI component corresponds to a three-dimensional UI component, and the VR display data corresponding to the reference display interface is determined based on the three-dimensional UI components and the display data.

[0069] In an embodiment of the present invention, a 3D UI component is obtained by performing the above-mentioned three-dimensional processing on the UI component. Specifically, before the UI component is processed in three dimensions, it is necessary to determine the value of the UI component to be processed. The value of the UI component to be processed may be information such as the type, size, and stereoscopic data after modeling of the UI component to be processed. The UI component to be processed may be a UI component that is prepared for three-dimensional processing by the host system after analyzing the user's sight habits. More specifically, the type of the UI component to be processed may be classified. For example, it may be determined according to the user's sight habits whether the UI component to be processed contains disease data that the user needs to know, and then the host system determines whether the UI component to be processed needs to be processed in three dimensions, or adjusts the display level of the UI component to be processed. For example, according to the user's sight data, it is analyzed that the attention to a certain UI component to be processed is not high, then the size of the UI component to be processed may be reduced, and the color may be lightened, thereby achieving the purpose of reducing the display level of the UI component to be processed. Differentiated three-dimensional processing of UI components through sight data can reduce the load of the three-dimensional processing on the server and improve the efficiency of three-dimensionalization of the UI component to be processed. The UI component to be processed can also be made into a three-dimensional UI component of a size that meets the user's visual habits by calculating the stereoscopic data of the UI component to be processed and the above-mentioned sight line data, and according to the proportional size of the reference display interface in the human eye.

[0070] According to the display data of the UI component in the disease display terminal, the server can process the display data, and the above display data can be a two-dimensional road disease data. The above display data is mapped by adding an additional depth dimension to it. The above depth dimension can be understood as a new coordinate vector that is perpendicular to the two-dimensional coordinates, and is mapped to the corresponding three-dimensional UI component after three-dimensional processing. Specifically, it is assumed that the coordinates of the above display data in the disease display terminal are (x, y), the above x is the horizontal coordinate, and y is the vertical coordinate. After receiving the coordinates of the above display data, the host system uses the processed line of sight data as the z coordinate corresponding to the above display data on the three-dimensional UI component based on the above user line of sight data, and the above z coordinate is the depth coordinate. Therefore, according to the above processing operation, the new coordinates (x, y, z) corresponding to the above display data on the three-dimensional UI component can be obtained. Finally, based on the new coordinates, the above display data is mapped to the corresponding three-dimensional UI component to form a VR display data. The VR display data obtained in this way conforms to the user's visual habits and optimizes the user's intuitive feeling of viewing the disease data in the reference display interface.

[0071] Optionally, in the step of performing three-dimensional processing on the UI component according to the line of sight data to obtain a three-dimensional UI component corresponding to the UI component, it is also possible to determine the user's line of sight weight for each UI component according to the line of sight data, determine the three-dimensional shape parameters of the UI component based on the shape of each UI component, determine the three-dimensional dynamic parameters of each UI component according to the line of sight weight, and determine the three-dimensional UI component corresponding to each UI component based on the three-dimensional shape parameters and the three-dimensional dynamic parameters.

[0072] In an embodiment of the present invention, the above-mentioned three-dimensional UI component is a dynamic three-dimensional UI component. The dynamic three-dimensional UI component may include three-dimensional dynamic parameters. The three-dimensional dynamic parameters refer to the position, direction, and moving speed of the corresponding three-dimensional UI component that changes dynamically during VR display.

[0073] The eye image sequence of the user in the reference display interface within a period of time is obtained through the above-mentioned face image acquisition device, and the eye sight in the above-mentioned eye image sequence is dynamically captured through the eye algorithms such as Optical Flow, Active Appearance Models, DEAM, ACF, etc. The sight weight and sight data are calculated according to the capture results, so as to determine the three-dimensional dynamic parameters and three-dimensional shape parameters of the UI component. The above-mentioned three-dimensional dynamic parameters can be the position, direction, moving speed, etc. of the three-axis coordinates of x, y, and z in space. The above-mentioned sight data can be used to determine the user's sight habits, and the three-dimensional shape parameters of the UI component are designed according to the user's sight habits.

[0074] In a possible embodiment, the eye image sequence can be analyzed to obtain the gaze dwell time, which can be the time the user's gaze stays on each UI component, and the gaze weight of the UI component can be calculated based on the gaze dwell time. Specifically, the gaze weight calculation formula can be based on the formula:

[0075]

[0076] Where W i represents the sight weight corresponding to the i-th UI component, T0 represents the time the user watches the reference display interface, T i Indicates the time the user's gaze stays on the i-th UI component, s i represents the area of ​​the ith UI component in the reference interface, (x i ,y i ) represents the average coordinate of the user's line of sight on the i-th UI component, (x0, y0) represents the content center coordinate of the i-th UI component, where the above T i and (x i ,y i) can be obtained based on the line of sight data, the above T0 can be measured, and the above s i And (x0, y0) can be obtained according to the underlying data corresponding to the reference display interface. It can be seen from the formula that the longer the user's sight stays on the UI component and the closer the sight point is to the content center of the UI component, the greater the sight weight of the UI component. The greater the sight weight of the UI component, the more attention the user pays to the UI component.

[0077] The higher the sight line weight of a UI component, the more it indicates that the UI component is a UI component that the user highly concentrates on, which further indicates that the UI component contains disease data that the user needs to understand carefully. Therefore, when the UI component is three-dimensional, it is required that the UI component can respond quickly to the user's observation, which means that the above-mentioned three-dimensional dynamic parameters of the UI component need to be adjusted. For example, the higher the sight line weight of the UI component, the smaller the movement speed of the corresponding three-dimensional UI component, making it easier for users to observe the three-dimensional UI component.

[0078] In a possible embodiment, the server obtains the shape of the UI component in the disease display terminal, and determines the three-dimensional shape parameters of the three-dimensional UI component in the reference display interface according to the shape features of the obtained UI component. The shape features of the above UI component can be obtained by analyzing the shape of the two-dimensional UI component on the screen of the disease display terminal through an image processing algorithm. Specifically, the two-dimensional shape features of the UI component are analyzed according to the image processing algorithm, including data such as size, angle, and roundness. And the three-dimensional shape is designed by adding an additional depth dimension to these two-dimensional shape features. The above depth dimension can be understood as a new coordinate vector that is perpendicular to the two-dimensional coordinate. And based on the above user line of sight data, the processed line of sight data is used as the depth dimension coordinate corresponding to the above display data on the three-dimensional UI component. For example, based on the fact that the two-dimensional shape is circular, the UI component can be made into a sphere when it is three-dimensionalized, and the corresponding data can be mapped on the surface of the sphere.

[0079] Optionally, in the step of determining the user's sight weight for each UI component based on the sight data, the number of times the user looks at each UI component and the length of time the sight stays can also be determined based on the sight data, and the user's sight weight for each UI component can be determined based on the number of times the sight is looked at and the length of time the sight stays.

[0080] In the embodiment of the present invention, the server determines the user's seriousness in browsing the content of this UI component by analyzing the user's sight data. The more times the user's sight contacts this UI component and the longer the sight stays, the more serious the user is in browsing the content of this UI component, which also indicates the importance of this UI component in viewing disease data.

[0081] The above-mentioned number of gaze times may be obtained by collecting the gaze data of the above-mentioned user through the above-mentioned face collection device. The above-mentioned gaze data may include the number of user gaze contacts and the length of time the user's gaze stays, etc., and by comparing the gaze data corresponding to each UI component, the gaze weight of the UI component is determined according to the gaze data, and the layout of the UI component in the reference display interface is designed as a reference according to the gaze weight.

[0082] The above-mentioned gaze dwelling time can be obtained based on the number of gaze contacts, which can be specifically calculated according to the following formula:

[0083] T=∑[S(A,B i )-L(C,B i )]

[0084] In the above formula, T represents the time from the first time the user's sight contacts the UI component to the first time the user leaves the UI component (the length of time the sight stays), and the above S(A,B i ) represents the i-th UI component B i At time point A, the user's line of sight is obtained, L(C,B i ) represents the i-th UI component B i The user's line of sight is lost at time point C. By recording the time from when the user's line of sight is acquired to when the user's line of sight is lost, the above-mentioned line of sight dwelling duration can be obtained once. The above-mentioned line of sight dwelling duration is obtained by accumulating the time when the user's line of sight dwells on the same UI component multiple times. More specifically, the number of line of sight contacts can also be determined through the above formula.

[0085] In an embodiment of the present invention, the sight weight of a UI component can be calculated by the following formula:

[0086]

[0087] Where W represents the sight weight, T represents the sight duration, N represents the number of sight contacts in the same UI component, and n represents the number of UI components. From the formula, we can see that when n is fixed, the larger T and N are, the larger w is; when T and N are fixed, the smaller n is, the larger w is. Therefore, by obtaining the sight duration and the number of UI components, the longer the sight duration is, the higher the weight of the UI component is, which proves that the UI component is a UI component that the user highly concentrates on, indicating that the UI component contains disease data that the user needs to understand carefully.

[0088] Optionally, in the step of determining VR display data corresponding to the reference display interface based on the three-dimensional UI component and the display data, the display data can also be classified to determine the target display data that needs to be three-dimensionally processed, and the target display data is three-dimensionally processed based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component; and the VR display data is determined based on the three-dimensional UI component and the corresponding three-dimensional target data.

[0089] In an embodiment of the present invention, in addition to performing three-dimensional processing on the above-mentioned UI component to obtain the above-mentioned three-dimensional UI component, it is also necessary to perform three-dimensional processing on the corresponding UI component display data and map it to the corresponding three-dimensional UI component to form a VR display data.

[0090] Before performing three-dimensional processing, the server can classify UI components according to the above-mentioned sight weight analysis results by processing the sight weight analysis results through a component classification algorithm. Specifically, the target UI component can be obtained through a decision tree algorithm, a logistic regression algorithm, etc. The above-mentioned target UI component can be the UI component that needs to be three-dimensionalized according to user needs based on the sight weight analysis results.

[0091] The display data can be divided into target display data and non-target display data. The non-target display data may be data that does not need to be displayed in three dimensions, and specifically may be display data of UI components that do not need to be three-dimensionalized. The target display data may be display data corresponding to the target UI component. When the UI component is set to the target UI component, it means that the user can view the disease data he wants to know through the target UI component. Therefore, when performing VR display, it is necessary to perform three-dimensional processing on the target display data corresponding to the target UI component to obtain three-dimensional target data, and map the three-dimensional target data to the corresponding three-dimensional UI component through the server, and determine the VR display data based on the target UI component and the three-dimensional target data.

[0092] Optionally, when performing three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component, it also includes determining a first three-dimensional shape parameter and a first three-dimensional dynamic parameter of the three-dimensional UI component, determining a second three-dimensional shape parameter and a second three-dimensional dynamic parameter of the corresponding target display data based on the first three-dimensional shape parameter and the first three-dimensional dynamic parameter, and determining the three-dimensional target data corresponding to the three-dimensional UI component based on the second three-dimensional shape parameter and the second three-dimensional dynamic parameter.

[0093] In the embodiment of the present invention, the second three-dimensional shape parameter and the second three-dimensional dynamic parameter of the corresponding target display data may be determined by acquiring the first three-dimensional shape parameter and the first three-dimensional dynamic parameter of the three-dimensional UI component.

[0094] Specifically, the second three-dimensional shape parameter can be obtained by proportionally modifying the first three-dimensional shape parameter. For example, on a disease display terminal, the UI component for displaying road cracks contains a text introduction to road cracks. The text introduction is written in a text box and occupies 50% of the page of the UI component. When performing three-dimensional processing, it should also be performed based on this ratio to obtain a three-dimensional text box introducing road cracks in the three-dimensional UI component, and the three-dimensional text box occupies 50% of the three-dimensional UI component interface. In this way, a VR display data with a normal proportion and capable of viewing disease information can be obtained.

[0095] In a possible embodiment, after the UI component is three-dimensionalized, when the three-dimensional UI component is moved on the reference display interface through the above controller, the first three-dimensional dynamic parameters of the three-dimensional UI component need to be obtained in advance, such as the speed of the three-dimensional UI component when the x-axis, y-axis, and z-axis are moved parallel to each other on the reference display interface, or the speed of the three-dimensional UI component when the x-axis, xz-axis, and yz-axis are moved at an angle. And according to the first three-dimensional dynamic parameters obtained, the three-dimensional target data corresponding to the three-dimensional UI component is also set with the same three-dimensional dynamic parameters, so that when the three-dimensional UI component is moved on the reference display interface, the three-dimensional target data will also move according to the corresponding speed, so that the three-dimensional UI component and the three-dimensional target data can form a whole and obtain a complete VR display data.

[0096] It should be noted that the VR display method for road damage provided by the embodiment of the present invention can be applied to shooting equipment, smart phones, computers, servers and other devices that can perform VR display of road damage.

[0097] like Figure 2 As shown, an embodiment of the present invention provides a VR display device for road damage, and the VR display device for road damage includes:

[0098] The acquisition module 201 is used to acquire the user's sight line data for the reference display interface, and acquire the UI components of the reference display interface and the display data corresponding to the UI components;

[0099] A determination module 202, configured to determine VR display data corresponding to the reference display interface based on the sight line data, the UI component, and the display data;

[0100] The display module 203 is used to display VR display data through VR devices.

[0101] Optionally, the acquisition module 201 includes:

[0102] A first adjustment submodule, used for adjusting the clarity of the reference display interface when the user browses the reference display interface;

[0103] A first acquisition submodule, used to acquire a facial image sequence of the user during the process of adjusting the definition of the reference display interface;

[0104] The first determination submodule is used to determine the sight line data of the user with respect to the reference display interface based on the facial image sequence.

[0105] Optionally, the determining module 202 includes:

[0106] A first processing submodule is used to perform three-dimensional processing on the UI component according to the sight line data to obtain a three-dimensional UI component corresponding to the UI component, where each UI component corresponds to one three-dimensional UI component;

[0107] The second determination submodule is used to determine VR display data corresponding to the reference display interface based on the three-dimensional UI component and the display data.

[0108] Optionally, the first processing submodule includes:

[0109] A first determining unit, configured to determine a sight line weight of the user for each of the UI components according to the sight line data;

[0110] A first determining unit, configured to determine a three-dimensional shape parameter of the UI component based on the shape of each of the UI components;

[0111] A second determining unit, configured to determine the three-dimensional dynamic parameters of each of the UI components according to the sight line weight;

[0112] A third determining unit is configured to determine a three-dimensional UI component corresponding to each of the UI components based on the three-dimensional shape parameters and the three-dimensional dynamic parameters.

[0113] Optionally, the first determining unit includes:

[0114] A first determining subunit is used to determine the number of times the user has gazed at each of the UI components and the duration of their gaze according to the gaze data;

[0115] The second determining subunit is used to determine the gaze weight of the user for each of the UI components based on the number of gaze times and the gaze stay duration.

[0116] Optionally, the first processing submodule further includes:

[0117] A fourth determining unit, configured to determine a first three-dimensional shape parameter and a first three-dimensional dynamic parameter of the three-dimensional UI component;

[0118] a fifth determining unit, configured to determine a second three-dimensional shape parameter and a second three-dimensional dynamic parameter corresponding to the target display data based on the first three-dimensional shape parameter and the first three-dimensional dynamic parameter;

[0119] A sixth determining unit is configured to determine three-dimensional target data corresponding to the three-dimensional UI component based on the second three-dimensional shape parameter and the second three-dimensional dynamic parameter.

[0120] It should be noted that the VR display device for road damage provided in the embodiment of the present invention can be applied to shooting equipment, smart phones, computers, servers and other equipment that can perform VR display of road damage.

[0121] The VR display device for road damage provided by the embodiment of the present invention can implement each process implemented by the VR display method for road damage in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0122] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program of a VR display method for road damage stored in the memory 302 and executable on the processor 301, wherein:

[0123] The processor 301 is used to call the computer program stored in the memory 302 and execute the following steps:

[0124] Acquire user's sight line data for a reference display interface, and acquire UI components of the reference display interface and display data corresponding to the UI components;

[0125] Determine VR display data corresponding to the reference display interface based on the line of sight data, the UI component, and the display data;

[0126] VR display data is displayed through VR devices.

[0127] Optionally, the acquiring of the user's sight line data for the reference display interface performed by the processor 301 includes:

[0128] When the user browses the reference display interface, adjusting the clarity of the reference display interface;

[0129] In the process of adjusting the definition of the reference display interface, obtaining a facial image sequence of the user;

[0130] Determining the user's line of sight data for the reference display interface based on the facial image sequence. Optionally, the processor 301 determines the VR display data corresponding to the reference display interface based on the line of sight data, the UI component, and the display data, including:

[0131] Performing three-dimensional processing on the UI component according to the sight line data to obtain a three-dimensional UI component corresponding to the UI component, wherein each UI component corresponds to one three-dimensional UI component;

[0132] VR display data corresponding to the reference display interface is determined based on the three-dimensional UI component and the display data.

[0133] Optionally, the processor 301 performs three-dimensional processing on the UI component according to the line of sight data to obtain a three-dimensional UI component corresponding to the UI component, including:

[0134] Determining the sight line weight of the user for each of the UI components according to the sight line data;

[0135] Determining three-dimensional shape parameters of the UI components based on the shapes of the respective UI components;

[0136] Determining three-dimensional dynamic parameters of each of the UI components according to the sight line weight;

[0137] The three-dimensional UI component corresponding to each of the UI components is determined based on the three-dimensional shape parameters and the three-dimensional dynamic parameters.

[0138] Optionally, the processor 301 determines the sight line weight of the user for each UI component using the sight line data, including:

[0139] Determine the number of times the user has looked at each of the UI components and the duration of their looking according to the sight data;

[0140] The gaze weight of the user for each of the UI components is determined based on the number of gaze times and the duration of the gaze stay.

[0141] Optionally, the determining, by the processor 301, VR display data corresponding to the reference display interface based on the three-dimensional UI component and the display data includes:

[0142] Classifying the display data to determine target display data that needs to be three-dimensionally processed;

[0143] Performing three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component;

[0144] Based on the three-dimensional UI component and the corresponding three-dimensional target data, VR display data is determined.

[0145] Optionally, the processor 301 performs three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component, including:

[0146] Determine a first three-dimensional shape parameter and a first three-dimensional dynamic parameter of the three-dimensional UI component;

[0147] Determine a second three-dimensional shape parameter and a second three-dimensional dynamic parameter corresponding to the target display data based on the first three-dimensional shape parameter and the first three-dimensional dynamic parameter;

[0148] The three-dimensional target data corresponding to the three-dimensional UI component is determined based on the second three-dimensional shape parameter and the second three-dimensional dynamic parameter.

[0149] It should be noted that the electronic device provided in the embodiment of the present invention can be applied to smart phones, computers, servers and other devices that can perform VR display methods for road hazards.

[0150] The electronic device provided in the embodiment of the present invention can implement each process implemented by the VR display method of road damage in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0151] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for VR display of road damage or the method for VR display of road damage on the application side provided in the embodiment of the present invention is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0153] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A VR display method for road damage, characterized in that: The method comprises the following steps: When the user browses the reference display interface, the clarity of the reference display interface is adjusted; the reference display interface displays the road disease data required by the user; In the process of adjusting the definition of the reference display interface, a sequence of eye images of the user on the reference display interface over a period of time is acquired through a face image acquisition device; The eye image sequence is processed by an eye algorithm, and the eye focus point is dynamically captured to obtain dynamic sight data of the user on the reference display interface. The eye focus point is the position where the eye focus stays on a certain disease display in the reference display interface when the user is relaxed; and obtaining a UI component of the reference display interface and display data corresponding to the UI component; Determine the sight line weight of the user for each UI component according to the sight line data; the sight line weight calculation formula is: Where W i represents the sight weight corresponding to the i-th UI component, T0 represents the time the user watches the reference display interface, T i Indicates the time the user's gaze stays on the i-th UI component, s i represents the area of ​​the ith UI component in the reference interface, (x i ,y i ) represents the average coordinate of the user's line of sight on the i-th UI component, (x0, y0) represents the content center coordinate of the i-th UI component, where T i and (x i ,y i ) is obtained based on the line of sight data. The above T0 is obtained by measurement, s i and (x0, y0) are obtained according to the underlying data corresponding to the reference display interface; Determining three-dimensional shape parameters of the UI components based on the shapes of the respective UI components; Determining three-dimensional dynamic parameters of each of the UI components according to the sight line weight; Determine, based on the three-dimensional shape parameters and the three-dimensional dynamic parameters, a three-dimensional UI component corresponding to each of the UI components, wherein each of the UI components corresponds to one three-dimensional UI component; Determining VR display data corresponding to the reference display interface based on the three-dimensional UI component and the display data; VR display data is displayed through VR devices.

2. The VR display method of road damage according to claim 1, characterized in that: Determining the sight line weight of the user for each of the UI components according to the sight line data includes: Determine the number of times the user has looked at each of the UI components and the duration of their looking according to the sight data; The gaze weight of the user for each of the UI components is determined based on the number of gaze times and the duration of the gaze stay.

3. The VR display method of road damage according to claim 1, characterized in that: The determining, based on the three-dimensional UI component and the display data, VR display data corresponding to the reference display interface includes: Classifying the display data to determine target display data that needs to be three-dimensionally processed; Performing three-dimensional processing on the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component; Based on the three-dimensional UI component and the corresponding three-dimensional target data, VR display data is determined.

4. The VR display method of road damage according to claim 3, characterized in that: The three-dimensional processing of the target display data based on the three-dimensional UI component to obtain three-dimensional target data corresponding to the three-dimensional UI component includes: Determine a first three-dimensional shape parameter and a first three-dimensional dynamic parameter of the three-dimensional UI component; Determine a second three-dimensional shape parameter and a second three-dimensional dynamic parameter corresponding to the target display data based on the first three-dimensional shape parameter and the first three-dimensional dynamic parameter; The three-dimensional target data corresponding to the three-dimensional UI component is determined based on the second three-dimensional shape parameter and the second three-dimensional dynamic parameter.

5. A VR display device for road damage, characterized in that: The VR display device for road damage includes: An acquisition module is used to adjust the clarity of the reference display interface when the user browses the reference display interface; the reference display interface displays the road disease data required by the user; in the process of adjusting the clarity of the reference display interface, an eye image sequence of the user in the reference display interface within a period of time is acquired through a face image acquisition device; the eye image sequence is processed through an eye algorithm, and the eye focus point is dynamically captured to acquire the user's dynamic sight line data for the reference display interface, where the eye focus point is the position where the user's eyes focus on a disease display in the reference display interface when the user is relaxed, and the UI components of the reference display interface and the display data corresponding to the UI components are acquired; A determination module is used to determine the sight weight of the user for each UI component according to the sight data; the sight weight calculation formula is: Where W i represents the sight weight corresponding to the i-th UI component, T0 represents the time the user watches the reference display interface, T i Indicates the time the user's gaze stays on the i-th UI component, s i represents the area of ​​the ith UI component in the reference interface, (x i ,y i ) represents the average coordinate of the user's line of sight on the i-th UI component, (x0, y0) represents the content center coordinate of the i-th UI component, where T i and (x i ,y i ) is obtained based on the line of sight data. The above T0 is obtained by measurement, s i and (x0, y0) are obtained according to the underlying data corresponding to the reference display interface; based on the shape of each UI component, the three-dimensional shape parameters of the UI component are determined; according to the sight weight, the three-dimensional dynamic parameters of each UI component are determined; based on the three-dimensional shape parameters and the three-dimensional dynamic parameters, the three-dimensional UI components corresponding to each UI component are determined, and each UI component corresponds to one three-dimensional UI component; based on the three-dimensional UI components and the display data, the VR display data corresponding to the reference display interface is determined; The display module is used to display VR display data through VR devices.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the VR display method for road damage as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the VR display method for road damage according to any one of claims 1 to 4 are implemented.

Images