District transformation method and device, computer equipment and storage medium

Abstract

The invention provides a district transformation method and device, computer equipment and a storage medium, and can determine the watching percentage of a user for key space elements in a plurality of districts according to eye movement data of the test user in the process of watching a panoramic video of the district, and then determine the district transformation rate based on electroencephalogram data and the watching percentage of the key space elements. And determining a prediction model corresponding to each area, and determining a transformation guidance scheme of the plurality of areas based on the prediction model corresponding to each area. Through the method, the evaluation result of each area can be quantified through the eye movement data and the electroencephalogram data in the panoramic video of the area watched by the user, so that the accurate recognition of the spatial perception characteristics of the area is realized, and quantitative guidance is provided for area transformation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a district reconstruction method and device, computer equipment and storage medium. BACKGROUND

[0002] Under the background of the integration of tourism and culture, the demand for the reconstruction of historical and cultural blocks is large, and spatial assessment is a necessary pre-work. The perception part of the existing spatial assessment system is mostly obtained by questionnaire survey or interview to obtain the reconstruction opinions of users on the block to be reconstructed. However, the questionnaire survey or interview has strong subjectivity, and cannot accurately measure the dynamic perception of users in different environments of the block to be reconstructed, so the evaluation results are more generalized and one-sided, and it is difficult to guide the actual design and optimization of historical blocks. SUMMARY

[0003] Therefore, the present application provides a district reconstruction method and device, computer equipment and storage medium.

[0004] In a first aspect, the present application provides a district reconstruction method, comprising:

[0005] obtaining panoramic videos of multiple districts, and obtaining electroencephalogram data and eye movement data of multiple test users when watching the panoramic videos;

[0006] determining key spatial elements of the multiple districts and gaze percentages corresponding to the key spatial elements respectively based on the eye movement data; wherein the gaze percentage is used to represent the visual attention degree of the test users to the key spatial elements;

[0007] determining prediction models corresponding to each district respectively based on the electroencephalogram data of each district and the gaze percentages of the key spatial elements; wherein the prediction model is used to represent the influence of the gaze percentage of the key spatial elements in the district on the electroencephalogram data;

[0008] determining reconstruction guidance schemes of the multiple districts based on the prediction models corresponding to each district respectively, wherein the reconstruction guidance schemes include update and adjustment schemes of the key spatial elements.

[0009] In a possible implementation, the determination of the key spatial elements of the multiple districts based on the eye movement data comprises:

[0010] obtaining perception questionnaire scores of the multiple districts, and determining a first sorting result of the perception intensity of the multiple districts according to the perception questionnaire scores;

[0011] determining the gaze percentages of each spatial element in each district;

[0012] For any spatial element, based on the gaze percentage of the spatial element in each region, a second ranking result of the visual attention degree of the spatial element in each region is determined;

[0013] Based on the first ranking result and the second ranking result, the key spatial elements of the multiple regions are determined.

[0014] In a possible implementation, the determination of the gaze percentage of each spatial element in each region includes:

[0015] For the panoramic video of any region, the panoramic video is divided into multiple scene videos according to a preset window;

[0016] For any scene video, based on the eye movement data of the corresponding moment of the scene video and the semantic segmentation result of the scene video, the gaze percentage of each spatial element in the scene video is determined;

[0017] Based on the gaze percentage of each spatial element in the multiple scene videos, the gaze percentage of each spatial element in the any region is determined.

[0018] In a possible implementation, the determination of the gaze percentage of each spatial element in the scene video based on the eye movement data of the corresponding moment of the scene video and the semantic segmentation result of the scene video includes:

[0019] For any scene video, based on the eye movement data of the corresponding moment of the scene video, gaze point information of the corresponding moment of the scene video is determined; wherein the gaze point information includes the position coordinates of the gaze point;

[0020] The any scene video is frame extracted, and the extracted video frame is subjected to semantic segmentation to determine the spatial elements contained in the video frame;

[0021] The gaze point information is projected to the video frame after semantic segmentation, and based on the projection result, the gaze percentage of each spatial element in the scene video is determined.

[0022] In a possible implementation, the determination of the key spatial elements of the multiple regions based on the first ranking result and the second ranking result includes:

[0023] The spatial elements with the same or opposite second ranking result corresponding to the first ranking result are determined as the key spatial elements;

[0024] If the spatial elements with the same or opposite second ranking result corresponding to the first ranking result exceed a preset number, the key spatial elements are screened based on at least one of the following conditions:

[0025] A proportion of the spatial element in the video picture, a frequency of appearance of the spatial element, whether it belongs to a preset spatial element.

[0026] In a possible implementation, the prediction model corresponding to each region is determined based on the electroencephalogram data of each region and the gaze percentage of the key spatial element.

[0027] The panoramic video of any region is divided into a plurality of scene videos according to a preset window.

[0028] The electroencephalogram wave index value of the electroencephalogram data corresponding to each scene video is determined.

[0029] The prediction model corresponding to any region is determined based on the electroencephalogram wave index value corresponding to each scene video and the gaze percentage of each key spatial element in each scene video.

[0030] In a possible implementation, the reconstruction guidance scheme of the plurality of regions is determined based on the prediction model corresponding to each region.

[0031] For any region, the average value of the electroencephalogram wave index of the region is determined based on the electroencephalogram wave index value corresponding to each scene video in the region.

[0032] The reference region is determined based on the average value of the electroencephalogram wave index of each region.

[0033] The reconstruction guidance scheme of the regions other than the reference region is determined based on the gaze percentage of each key spatial element in the prediction model corresponding to the reference region.

[0034] In a second aspect, the present application provides a region reconstruction device, which comprises:

[0035] The acquisition module is configured to acquire panoramic videos of a plurality of regions and acquire electroencephalogram data and eye movement data of a plurality of test users when the test users watch the panoramic videos.

[0036] The first determination module is configured to determine key spatial elements of the plurality of regions and gaze percentages corresponding to the key spatial elements based on the eye movement data, wherein the gaze percentage is used to represent the degree of visual attention of the test users to the key spatial elements.

[0037] The second determination module is configured to determine a prediction model corresponding to each region based on the electroencephalogram data of each region and the gaze percentage of the key spatial element, wherein the prediction model is used to represent the influence of the gaze percentage of the key spatial element in the region on the electroencephalogram data.

[0038] A third determining module is configured to determine a reconstruction guidance scheme of the multiple regions based on the prediction model corresponding to each region, wherein the reconstruction guidance scheme includes an update adjustment scheme of the key spatial element.

[0039] In a third aspect, a computer readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements the region reconstruction method.

[0040] In a fourth aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the region reconstruction method when executing the program.

[0041] The region reconstruction method, device, computer device and storage medium provided by the present application can determine the fixation percentage of the user for the key spatial element in the multiple regions according to the eye movement data in the process of the user watching the panoramic video of the region, and then determine the prediction model corresponding to each region based on the electroencephalogram data and the fixation percentage of the key spatial element, and determine the reconstruction guidance scheme (i.e. the update adjustment scheme) of the multiple regions based on the prediction model corresponding to each region. Through this method, the evaluation results of each region can be quantified by the eye movement data and the electroencephalogram data in the process of the user watching the panoramic video of the region, and the accurate identification of the spatial perception characteristics of the region can be realized, thereby providing quantitative guidance for the reconstruction of the region. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of a region reconstruction method according to an example embodiment of the present application;

[0043] Figure 2 is a flowchart of a method for determining a key spatial element according to an example embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a projection result according to an example embodiment of the present application;

[0045] Figure 4 is an example diagram of a key spatial element determination and application process according to an example embodiment of the present application;

[0046] Figure 5 is an architectural schematic diagram of a region reconstruction device according to an example embodiment of the present application;

[0047] Figure 6 is a structural schematic diagram of a computer device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0048] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description of the exemplary embodiments is intended to apply to various alternative embodiments as well. The following description is not limited to the exemplary embodiments, but rather, is applicable to any apparatus and / or method which falls within the scope of the present application. Although specific arrangements were described herein, other arrangements can be utilized and the exemplary embodiments should not be construed as limited to the arrangements described herein, but only by the claims and their equivalents.

[0049] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used only to distinguish one from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present application. As used herein, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" depending on the context.

[0051] Under the background of the integration of tourism and culture, the demand for the renovation of historical and cultural blocks is large, and spatial assessment is a necessary pre-work. The perception part of the existing spatial assessment system is mostly obtained through questionnaire survey or interview to obtain the users' opinions on the renovation of the block to be transformed. However, due to the strong subjectivity of the questionnaire survey or interview, and the inability to accurately measure the dynamic perception of the user in the different environment of the block to be transformed, the evaluation result is more generalized and one-sided, and it is difficult to guide the actual design and optimization of the historical block.

[0052] Based on this, the present application provides a block renovation method, device, computer equipment and storage medium, which can determine the fixation percentage of the user for the key space elements in the multiple blocks according to the eye movement data in the process of the user watching the panoramic video of the block, and then determine the prediction model corresponding to each block based on the electroencephalogram data and the fixation percentage of the key space elements, and determine the renovation guidance scheme of the multiple blocks based on the prediction model corresponding to each block. Through this method, the evaluation results of each block can be quantified by the eye movement data and electroencephalogram data in the panoramic video of the user watching the block, and the accurate identification of the spatial perception characteristics of the block is realized, providing quantitative guidance for the block renovation.

[0053] Referring to Figure 1As shown, a flowchart of a piece area reconstruction method provided by the present application includes the following steps:

[0054] S101, acquire panoramic videos of multiple piece areas, and acquire electroencephalogram data and eye movement data of multiple test users when watching the panoramic videos.

[0055] S102, based on the eye movement data, determine key space elements of the multiple piece areas and the fixation percentage corresponding to each key space element respectively; wherein the fixation percentage is used to represent the visual attention degree of the test users to the key space elements.

[0056] S103, based on the electroencephalogram data of each piece area and the fixation percentage of the key space elements, determine the prediction model corresponding to each piece area respectively; wherein the prediction model is used to represent the influence of the fixation percentage of the key space elements in the piece area on the electroencephalogram data.

[0057] S104, based on the prediction model corresponding to each piece area respectively, determine the reconstruction guidance scheme of the multiple piece areas, wherein the reconstruction guidance scheme includes the update adjustment scheme of the key space elements.

[0058] The following is a detailed description of the above steps.

[0059] For S101,

[0060] Here, the piece area can be understood as an area composed of multiple streets, and a piece area can include multiple streets. The multiple piece areas can be multiple adjacent blocks belonging to the same block. For example, the multiple piece areas can include core protection range, construction control zone, landscape coordination area and other types of piece areas.

[0061] The eye movement data can include the three-dimensional coordinates of the first fixation point, the three-dimensional vector of the fixation direction, and the confidence of the data.

[0062] For S102,

[0063] Each piece area contains multiple space elements such as walls, plants, and paving, which together constitute the spatial structure and visual perception of the block. Each space element not only has an impact on the functionality and accessibility of the space, but also plays different roles in visual perception and attracts the attention of observers. The key space element is a space element with high importance and potential correlation with brain activity selected from multiple preset space elements.

[0064] In one possible implementation, when determining the key space elements of the multiple piece areas based on the eye movement data, the method shown in Figure 2 includes the following steps:

[0065] S201, obtain perception questionnaire scores of the plurality of subareas, and determine a first ranking result of perception intensities of the plurality of subareas according to the perception questionnaire scores.

[0066] Here, the perception questionnaire scores can be scores of questionnaires filled by the test users after watching the panoramic video, and the perception questionnaire scores can reflect subjective psychological perception of the test users on the subarea environment.

[0067] Here, the first ranking result can refer to a ranking result obtained by arranging the perception questionnaire scores of each subarea in descending order.

[0068] S202, determine a fixation percentage of each spatial element in each subarea.

[0069] S203, for any spatial element, determine a second ranking result of visual attention degrees of the spatial element in each subarea based on the fixation percentages of the spatial element in each subarea.

[0070] Here, the second ranking result can be understood as a comprehensive ranking result. Specifically, for any spatial element, the fixation percentages of the spatial element in each subarea can be ranked in descending order, and the second ranking result can be obtained.

[0071] For example, if the fixation percentage of spatial element K in subarea A is 10%, the fixation percentage of spatial element K in subarea B is 11%, and the fixation percentage of spatial element K in subarea C is 5%, the second ranking result corresponding to spatial element K is subarea B > subarea A > subarea C.

[0072] S204, determine key spatial elements of the plurality of subareas based on the first ranking result and the second ranking result.

[0073] Specifically, spatial elements with the same or opposite corresponding second ranking results and first ranking results can be determined as key spatial elements. For example, if the first ranking result is subarea B > subarea A > subarea C, spatial elements with the corresponding second ranking result of subarea B > subarea A > subarea C or subarea B < subarea A < subarea C can be determined as key spatial elements.

[0074] Here, if the second ranking result corresponding to a spatial element is the same as the first ranking result, it means that the spatial element is a positive spatial element, and if the second ranking result corresponding to a spatial element is opposite to the first ranking result, it means that the spatial element is a negative spatial element.

[0075] Optionally, due to the excessive number of key spatial elements, the complexity of the district renovation guidance scheme may be affected, therefore, if the number of spatial elements corresponding to the second sorting result that is the same as or opposite to the first sorting result exceeds a preset number, the key spatial elements can be screened based on at least one of the following conditions:

[0076] The proportion of the spatial element in the video picture, the appearance frequency of the spatial element, whether it belongs to a preset spatial element.

[0077] Here, screening the key spatial elements based on the above conditions can mean screening the key spatial elements from the spatial elements corresponding to the second sorting result that is the same as or opposite to the first sorting result based on the above conditions.

[0078] The proportion of the spatial element in the video picture can mean the ratio between the number of pixel points corresponding to the spatial element and the total number of pixels of the video picture. Here, to reduce the amount of calculation, the panoramic video can be frame extracted, and then the average proportion of the spatial element in the video picture in the frame extracted video frame is determined.

[0079] The appearance frequency of the spatial element can mean the number of video frames containing the spatial element.

[0080] Optionally, in S202, when determining the gaze percentage of each spatial element in each district, since the panoramic video of the entire district is long, for the panoramic video of any district, the panoramic video can be divided into multiple scene videos according to a preset window size (the preset window may be 10 seconds for example); for any scene video, based on the eye movement data at the corresponding time of the scene video and the semantic segmentation result of the scene video, the gaze percentage of each spatial element in the scene video is determined, and then based on the gaze percentage of each spatial element in multiple scene videos of a district, the gaze percentage of each spatial element in the district is determined.

[0081] Among them, the eye movement data corresponding to the time of the scene video can be understood as the eye movement data collected in the time interval where the scene video is located.

[0082] For any scene video, when determining the gaze percentage of each spatial element in the scene video based on the eye movement data at the corresponding time of the scene video and the semantic segmentation result of the scene video, the gaze point information at the corresponding time of the scene video can be determined based on the eye movement data at the corresponding time of the scene video; wherein the gaze point information includes the position coordinates of the gaze point; then the any scene video is frame extracted, and the video frame extracted is subjected to semantic segmentation to determine the spatial element contained in the video frame; the gaze point information is projected to the video frame after semantic segmentation, and based on the projection result, the gaze percentage of each spatial element in the scene video is determined.

[0083] The duration of the scene video is the same as the preset window size mentioned above, for example, 10 seconds. The eye movement data at the corresponding moment in the scene video is also the eye movement data within 10 seconds. When determining the fixation point information at the corresponding moment in the scene video based on the eye movement data, since the eye movement data contains many first fixation points, the first fixation points in the eye movement data within the time window range can be merged first. The merged fixation point is used as the fixation point information at the corresponding moment in the scene video.

[0084] For example, when merging the first gaze points, it can be determined whether the merged gaze points meet the following requirements:

[0085] The time interval between two consecutive fixations is less than or equal to N frames (e.g., 3 frames), or the angular difference between the eye movement directions of two consecutive fixations is less than or equal to a preset degree (e.g., 0.5°).

[0086] Here, the fixation point information mainly includes the position coordinates of the merged second fixation point. This reduces the number of fixation points, making it easier to calculate the fixation percentage of spatial features.

[0087] Optionally, after acquiring the eye-tracking data for the corresponding moment in the scene video, the eye-tracking data can be preprocessed. This preprocessing includes extracting gaze angle features and further optimizing the data using vector rotation and angle calculation methods. Simultaneously, filtering methods such as median filtering can be used to reduce noise, and finally, the blink rate and pupil movement rate for each frame are calculated to extract more stable eye-tracking features.

[0088] For example, the gaze vector v represents the direction vector from the eye position to the gaze point, which can be calculated using the following formula:

[0089] v=(x fix -x eye y fix -y eye , z fix -z eye )

[0090] Among them, (x fix y fix , z fix (x) represents the eye position coordinates, which can be preset fixed values. eye y eye , z eye ) represents the coordinates of the gaze point.

[0091] The line-of-sight vector v and a reference direction (such as the reference direction v of the z-axis) ref The angle between (0, 0, 1) can be calculated using the following formula:

[0092]

[0093] Where v1·v2 represents the vector click, calculated as follows: v1·v2=x1x2+y1y2+z1z2.

[0094] ‖v1‖ and ‖v2‖ represent the magnitudes of the vectors, calculated as follows:

[0095] Based on the above formula, the angular difference between the eye movement directions of two consecutive fixation points can be calculated, and then it can be determined whether to merge the fixation points.

[0096] After semantic segmentation of the video frames extracted from the scene video, each region in the video frame carries its own corresponding semantic information (that is, it determines which spatial elements are contained in the extracted video frame). When projecting the gaze point information onto the semantically segmented video frame, the location coordinates of the second gaze point can be used to determine which spatial element's region each second gaze point falls within. For any spatial element in the video frame, the gaze percentage of that spatial element can be determined by the ratio of the number of second gaze points falling within the region corresponding to that spatial element to the total number of second gaze points.

[0097] For example, the projection result is as follows Figure 3 As shown, Figure 3 In the projection, different colored areas represent different spatial elements, and the points in the areas represent the second fixation points of the projection. Therefore, the fixation percentage of each spatial element can be determined through projection.

[0098] In another possible implementation, since the scene video is long, projecting it onto only one video frame may not be representative. Therefore, the scene video can be divided again, and then the fixation percentage of spatial elements in the divided video segments can be calculated. The mean of the fixation percentages can then be used as the fixation percentage of spatial elements in the scene video.

[0099] For example, if the duration of the scene video is 10 seconds, the scene video can be divided into two segments. One video frame is extracted from the first 5 seconds of the video segment (e.g., video frame 1), and one video frame is extracted from the last 5 seconds of the video segment (e.g., video frame 2). Then, the eye movement data corresponding to the scene video is also divided. The first fixation point in the eye movement data of the first 5 seconds is merged and projected onto video frame 1. The first fixation point in the eye movement data of the last 5 seconds is merged and projected onto video frame 2. Then, the fixation percentage 1 of spatial element x is determined based on the projection result of video frame 1, and the fixation percentage 2 of spatial element x is determined based on the projection result of video frame 2. Finally, the average of fixation percentage 1 and fixation percentage 2 is taken as the fixation percentage of spatial element x in this segment of the scene video.

[0100] In the gaze percentage of each spatial element in the multiple scene videos based on one area, when determining the gaze percentage of each spatial element in any area, the sum of the gaze percentage of each spatial element in the multiple scene videos can be taken as the gaze percentage of each spatial element in the area.

[0101] For example, if the panoramic video of an area includes N scene videos, for any spatial element, the sum of the gaze percentage of the spatial element in the N scene videos can be taken as the gaze percentage of the spatial element in the area.

[0102] Since the gaze percentage of a spatial element is determined by the number of gaze points falling in the area where the spatial element is located, the more gaze points falling in the area where the spatial element is located, the higher the gaze percentage of the spatial element, which means that the user gazes at the spatial element more frequently / time, and thus the gaze percentage of the spatial element can represent the degree of visual attention of the test user to the key spatial element.

[0103] For example, referring to Figure 4 As shown, for the panoramic video of each area, the video is divided into multiple scene videos according to a length of 10 seconds, including segment 1, segment 2, segment 3,..., and segment n. For each segment video, it is further divided into a front 5-second sub-segment video and a rear 5-second sub-segment video. For each sub-segment video, the gaze percentage of each spatial element can be determined based on image data (i.e., the video frames of the extracted frames) and eye movement data of the time period. Then, based on the gaze percentage of each spatial element in the front 5-second sub-segment video and the gaze percentage of each spatial element in the rear 5-second sub-segment video, the gaze percentage of each spatial element in the 10-second segment video is determined. Then, based on the ranking result of the perception questionnaire score (i.e., the first ranking result), key spatial elements with consistent trend with the ranking result of the perception questionnaire score are screened out, where the consistent trend includes the same or completely opposite ranking results. Then, based on the EEG data and the key spatial elements, a joint study is performed, and subsequent steps such as determining a prediction model are executed. The subsequent steps will be described below.

[0104] In this way, key spatial elements can be screened out. Since the second ranking result of the screened key spatial elements is the same as or opposite to the first ranking result of the perception questionnaire score, the first ranking result reflects the psychological perception level of the test user, and the second ranking result reflects the visual perception trend of the user, and thus the screened key spatial elements can be understood as spatial elements affecting the psychological perception and visual perception of the user.

[0105] The difference in the gaze percentage of spatial elements in different areas reveals the different points of attention of the observer to the street space during walking, and further reflects the changes in the function and visual appeal of the street space.

[0106] For S103,

[0107] The prediction model is used to characterize the influence of the gaze proportion of the key spatial elements in the region on the electroencephalogram data. The prediction model is a mathematical model. If the key spatial elements include six spatial elements represented by x1, x2, x3, x4, x5, and x6, the prediction model can be expressed as y = a x1 + b x2 + c x3 + d x4 + e x5 + f x6, where y represents a certain electroencephalogram wave index value of the electroencephalogram data. In actual application, the mathematical structure of the prediction model can be adjusted according to actual conditions, for example, it can be other nonlinear models, or it can be a linear model containing cross terms and constant terms.

[0108] In a possible implementation, when the prediction model corresponding to each region is determined based on the electroencephalogram data of each region and the gaze percentage of the key spatial elements, for the panoramic video of any region, the panoramic video can be divided into multiple scene videos according to a preset window; then the electroencephalogram wave index value of the electroencephalogram data corresponding to each scene video is determined; and then the prediction model corresponding to the any region is determined based on the electroencephalogram wave index value corresponding to each scene video and the gaze percentage of each key spatial element in each scene video.

[0109] Here, the electroencephalogram wave index value can be understood as a certain index representing the electroencephalogram data, which can be the value of the Gamma wave for example. Since the electroencephalogram data is a continuous data, the electroencephalogram wave index value of the scene video can be the value of the Gamma wave at a certain time of the electroencephalogram data.

[0110] For example, the electroencephalogram wave index value corresponding to a certain scene video can be the value of the Gamma wave in the time interval of the scene video.

[0111] For example, if a certain scene video is from the 10th second to the 20th second when determining the gaze percentage of the spatial elements, the value of the Gamma wave from the 10th second to the 20th second can be determined as the electroencephalogram wave index value corresponding to the scene video.

[0112] After the above processing, the obtained data set can be as shown in the following table (taking the key spatial elements including four spatial elements x1, x2, x3, and x4 as an example):

[0113]

[0114]

[0115] In the above table, n a m bThe bth scene video representing the ath subarea.

[0116] If the average Gamma value of the n1 subarea is y1, the average Gamma value of the n2 subarea is y2, and the average Gamma value of the n3 subarea is y3, the prediction models of the n1, n2, and n3 subareas can be determined by fitting as follows:

[0117] y1=a x1+b x2+c x3+d x4

[0118] y2=a x1 ’ +b x2 ’ +c x3 ’ +d x4 ’

[0119] y3=a x1 ” +b x2 ” +c x3 ” +d x4 ”

[0120] As can be seen, the values of the key spatial elements are different in different prediction models. Here, the values of the key spatial elements are not the values of the key spatial elements calculated in the subarea, but the values obtained after fitting. The values of the key spatial elements can be understood as the gaze percentages of the key spatial elements after fitting.

[0121] For S104,

[0122] The difference in the overall perception of different subareas can be explained by the gaze distribution of people on key spatial elements (such as walls, greenery, commercial external display, signs, people flow, and vehicles). The updated design of the space can regulate people's perception of the subarea by changing the visual presentation and proportion of each element, further optimize the environmental design, and improve the visual appeal and psychological comfort.

[0123] In a possible implementation, when determining the renovation guidance scheme of the plurality of subareas based on the prediction model corresponding to each subarea, for any subarea, the average value of the brain wave index of the subarea can be determined based on the brain wave index values corresponding to each scene video in the subarea; then the reference subarea is determined based on the average values of the brain wave indexes of the subareas; and the renovation guidance scheme of the subarea other than the reference subarea is determined based on the gaze percentages of the key spatial elements in the prediction model corresponding to the reference subarea.

[0124] Here, the brain wave index average value can be an average Gamma value, and the reference slice area can be a slice area with the highest average Gamma value. Gamma waves are closely related to advanced cognitive functions, perceptual integration, learning, and the speed of information processing, and are involved in the integration and processing of external information by the brain. The power of high-frequency Gamma waves is most significant in the differences in brain wave characteristics in different slice areas. Therefore, the influence of different regions on an individual's advanced cognitive functions (such as attention, information integration, and perceptual complexity) can be reflected through Gamma waves. The higher the average Gamma value, the better the user experience can be understood.

[0125] During the reconstruction of multiple slice areas, the perception results (i.e., average Gamma values) of users for each slice area can be adjusted by adjusting the gaze proportions of key spatial elements, thereby improving the perceptual consistency of the block. For example, if n1 slice area is the reference slice area, the gaze proportions of key spatial elements in n2 and n3 slice areas can be adjusted to make the average Gamma values of n2 and n3 slice areas closer to the average Gamma value of n1 slice area. The gaze proportions of key spatial elements can be adjusted by adjusting the spatial proportions of each spatial element.

[0126] For example, when determining the reconstruction guidance scheme for any slice area, the difference (such as Δx1, Δx2, Δx3, Δx4) between the values corresponding to each key spatial element in the prediction model of the slice area and the values corresponding to each key spatial element in the prediction model of the reference slice area can be determined, and then the reconstruction guidance scheme can be generated based on the difference (for example, the difference can be filled into a preset scheme template). Here, the value corresponding to the key spatial element is the gaze percentage corresponding to the key spatial element.

[0127] In practical applications, designers can change the visual forms, position layouts, size ratios, etc. of various elements to control the proportions of these elements in people's visual focus. Such adjustments can not only affect people's emotional responses to space, but also change their assessments of the functionality and aesthetics of space. For example, by increasing or decreasing the visual proportion of certain elements, the openness, decoration, interactivity, etc. of the space can be enhanced or weakened, thereby affecting people's overall perception. The reconstruction guidance scheme generated in the scheme provided by the present application can provide reconstruction suggestions for designers, such as adjusting the spatial proportions of various spatial elements to increase the gaze percentage corresponding to a key spatial element, thereby assisting designers in slice area reconstruction.

[0128] Corresponding to the foregoing embodiments of the slice area reconstruction method, the present application also provides an embodiment of a slice area reconstruction device.

[0129] Referring to Figure 5 As shown in the figure, an architecture schematic diagram of a slice area reconstruction device provided by the present application includes:

[0130] The acquisition module 501 is configured to acquire panoramic videos of a plurality of regions, and acquire electroencephalogram data and eye movement data of a plurality of test users when the test users watch the panoramic videos.

[0131] The first determination module 502 is configured to determine key spatial elements of the plurality of regions and gaze percentages corresponding to the key spatial elements respectively based on the eye movement data, wherein the gaze percentages are used to represent visual attention degrees of the test users to the key spatial elements.

[0132] The second determination module 503 is configured to determine a prediction model corresponding to each region based on electroencephalogram data of each region and the gaze percentages of the key spatial elements, wherein the prediction model is used to represent an influence of a gaze percentage of a key spatial element in a region on the electroencephalogram data.

[0133] The third determination module 504 is configured to determine a reconstruction guidance scheme of the plurality of regions based on the prediction model corresponding to each region, wherein the reconstruction guidance scheme includes an update adjustment scheme of the key spatial elements.

[0134] In a possible implementation, when determining the key spatial elements of the plurality of regions based on the eye movement data, the first determination module 502 is configured to:

[0135] acquire perception questionnaire scores of the plurality of regions, and determine a first ranking result of perception intensities of the plurality of regions according to the perception questionnaire scores;

[0136] determine gaze percentages of spatial elements in each region;

[0137] for any spatial element, determine a second ranking result of visual attention degrees to the spatial element in each region based on the gaze percentages of the spatial element in each region;

[0138] determine the key spatial elements of the plurality of regions based on the first ranking result and the second ranking result.

[0139] In a possible implementation, when determining the gaze percentages of the spatial elements in each region, the first determination module 502 is configured to:

[0140] for the panoramic video of any region, divide the panoramic video into a plurality of scene videos according to a preset window;

[0141] for any scene video, determine gaze percentages of spatial elements in the scene video based on eye movement data at a corresponding moment of the scene video and a semantic segmentation result of the scene video;

[0142] Determine the gaze percentage of each spatial element in the any region based on the gaze percentage of each spatial element in the plurality of scene videos.

[0143] In a possible implementation, for any scene video, the first determining module 502 is configured to:

[0144] For any scene video, determine the gaze point information of the corresponding time of the scene video based on the eye movement data of the corresponding time of the scene video; wherein the gaze point information comprises the position coordinates of the gaze point.

[0145] Frame the any scene video, and perform semantic segmentation on the extracted video frames to determine the spatial elements contained in the video frames.

[0146] Project the gaze point information to the video frames after semantic segmentation, and determine the gaze percentage of each spatial element in the scene video based on the projection result.

[0147] In a possible implementation, the first determining module 502, when determining the key spatial elements of the plurality of regions based on the first ranking result and the second ranking result, is configured to:

[0148] Determine the spatial elements with the same or opposite first ranking result as the corresponding second ranking result as the key spatial elements.

[0149] If the spatial elements with the same or opposite first ranking result as the corresponding second ranking result exceed a preset number, screen the key spatial elements based on at least one of the following conditions:

[0150] The proportion of the spatial element in the video picture, the appearance frequency of the spatial element, and whether it belongs to a preset spatial element.

[0151] In a possible implementation, the second determining module 503, when determining the prediction model corresponding to each region based on the electroencephalogram data of each region and the gaze percentage of the key spatial elements, is configured to:

[0152] For the panoramic video of any region, divide the panoramic video into a plurality of scene videos according to a preset window.

[0153] Determine the electroencephalogram wave index value of the electroencephalogram data corresponding to each scene video.

[0154] Fit the prediction model corresponding to the any region based on the electroencephalogram wave index value of the electroencephalogram data corresponding to each scene video, and the gaze percentage of each key spatial element in each scene video.

[0155] In a possible implementation, the third determining module 504, when determining the reconstruction guidance scheme of each region based on the prediction model corresponding to each region, is configured to:

[0156] For any region, determine the average value of the brainwave index of the region based on the brainwave index values corresponding to each scene video in the region.

[0157] Determine a reference region based on the average value of the brainwave index of each region.

[0158] Determine the reconstruction guidance scheme of each region except the reference region based on the gaze percentage of each key spatial element in the prediction model corresponding to the reference region.

[0159] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0160] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be referred to the part of the method embodiment. The device embodiments described above are only illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the present application. Those skilled in the art can understand and implement without creative labor.

[0161] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program can be used to execute the region reconstruction method described in the above embodiments.

[0162] The present application also provides a computer device, as shown in Figure 6 The structure diagram of the computer device provided by the present application is shown in the hardware layer, which includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course, it can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to realize the region reconstruction method described in the above embodiments. Of course, in addition to the software implementation, the present specification does not exclude other implementation modes, such as logic device or combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but also can be hardware or logic device.

[0163] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0164] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit), and the apparatus can be implemented as special purpose logic circuitry.

[0165] Computers suitable for the execution of a computer program include, by way of example, general and / or special purpose microprocessors, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The essential elements of a computer are a central processing unit for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0166] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0167] While the specification contains many specifics, these should not be construed as limiting the scope of any invention or of what can be claimed, but as merely providing illustrations of some of the embodiments of the inventions. Certain features that are, for clarity, described above in the context of separate embodiments can also be provided in combination in a single embodiment. Conversely, various features that are, for brevity, described above in the context of a single embodiment can also be provided separately or in any suitable subcombination. In addition, while features can be described above as being implemented in digital electronic circuitry, one or more features also can be implemented in analog circuitry, in embodiments of the application. As used herein, the term "circuitry" refers to all of the following: (a) hardware-only circuitry implementations (which can include combinations of custom circuits, ASICs, and other custom integrated circuits); (b) any implementations that include digital and / or analog hardware-firmware-software combinations; and (c) any implementations that include a combination of hardware, software, and / or firmware that all generally work together to produce the results of the technology described herein. In some embodiments, the hardware can include transistors and other circuitry that are configured to perform the functions of the technology described herein.

[0168] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such order, nor that all illustrated operations be performed, to implement the desired results. One will appreciate that many other operations can be performed between the described operations and in parallel with the described operations. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0169] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. In some cases, the processes depicted in the figures do not necessarily require the particular order illustrated, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing can be advantageous.

[0170] The above-described embodiments are merely meant to be illustrative of only the preferred embodiments of the present application and the present application is not to be limited to such embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the present application.

Claims

1. A method of urban renewal, characterized by, The method comprises: acquiring panoramic videos of multiple zones, and acquiring electroencephalogram data and eye movement data of multiple test users when the test users watch the panoramic videos; based on the eye movement data, determining key spatial elements of the multiple zones and a fixation percentage corresponding to each of the key spatial elements; wherein the fixation percentage is used to represent the degree of visual attention of the test users to the key spatial elements; based on the electroencephalogram data of each zone and the fixation percentage of the key spatial elements, determining a prediction model corresponding to each zone; wherein the prediction model is used to represent the influence of the fixation percentage of the key spatial elements in a zone on the electroencephalogram data; based on the prediction model corresponding to each zone, determining a reconstruction guidance scheme for the multiple zones, wherein the reconstruction guidance scheme includes an update adjustment scheme for the key spatial elements.

2. The method of claim 1, wherein, The method comprises: acquiring perception questionnaire scores of the multiple zones, and determining a first ordering result of the perception intensity of the multiple zones according to the perception questionnaire scores; determining the fixation percentage of each spatial element in each zone; for any spatial element, based on the fixation percentage of the spatial element in each zone, determining a second ordering result of the degree of visual attention to the spatial element in each zone; based on the first ordering result and the second ordering result, determining the key spatial elements of the multiple zones.

3. The method of claim 2, wherein, The method comprises: for the panoramic video of any zone, dividing the panoramic video into multiple scene videos according to a preset window; for any scene video, based on the eye movement data at the corresponding time of the scene video and the semantic segmentation result of the scene video, determining the fixation percentage of each spatial element in the scene video; based on the fixation percentage of each spatial element in the multiple scene videos, determining the fixation percentage of each spatial element in the any zone.

4. The method of claim 3, wherein, The method comprises: for any scene video, based on the eye movement data at the corresponding time of the scene video, determining the fixation point information at the corresponding time of the scene video; wherein the fixation point information includes the position coordinates of the fixation point; frame extraction is performed on the any scene video, and semantic segmentation is performed on the extracted video frames to determine the spatial elements contained in the video frames; the fixation point information is projected to the video frames after semantic segmentation, and based on the projection result, the fixation percentage of each spatial element in the scene video is determined.

5. The method of claim 2, wherein, The method comprises: spatial elements corresponding to the second ordering result that is the same as or opposite to the first ordering result are determined as the key spatial elements; wherein, if the spatial elements corresponding to the second ordering result that is the same as or opposite to the first ordering result exceed a preset number, the key spatial elements are screened based on at least one of the following conditions: A proportion of the spatial element in the video picture, a frequency of appearance of the spatial element, and whether the spatial element belongs to a preset spatial element.

6. The method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

7. The method of claim 6, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The device comprises:

8. A device for urban renewal, characterized in that, The device comprises: The device comprises: The program is executed by the processor to implement the steps of the method of any one of claims 1-7. The processor executes the program to implement the steps of the method of any one of claims 1-7. ​ 9. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​ 10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, ​

Images