Method, device, equipment, medium and program product for multi-dimensionally sensing state information of target area

Abstract

The invention provides a method, a device, equipment, a medium and a program product for multi-dimensionally sensing state information of a target area. Comprising the steps of determining a target image feature point based on a target image in response to the received geographical location information of a target point location sent by a terminal, the orientation information of the terminal relative to the target point location and the target image acquired by the terminal for the target point location; based on the geographic position information, the orientation information and the target image feature point, searching a database of the server to obtain historical scene information of the target point location; the historical scene information of the target point location is rendered, a historical image of the target point location is generated, and the historical image is sent to the terminal; obtaining visual difference information of the target point location based on the historical image and the target image by using a preset feature matching algorithm; and generating target state information based on the plurality of visual difference information and the sensory information of the plurality of target points.

Description

Technical Field

[0001] The present disclosure relates to the field of geographic information technology, and more specifically, to a method, apparatus, device, medium, and program product for multi-dimensionally perceiving the state information of a target area. Background Art

[0002] With the rapid development of global urbanization, the urban population has increased sharply and the scale has been continuously expanding. The urbanization rate has reached a relatively high level, and many cities are facing various pressures such as population, resources, and environment. The existing urban physical examination methods need to collect data of target points in the target urban area, and manually judge and score the currently collected data without historical data comparison, so that the evaluation score corresponding to the current state of the target point cannot be accurately determined, resulting in inaccurate urban physical examination results. Summary of the Invention

[0003] In view of this, the present disclosure provides a method, apparatus, device, medium, and program product for multi-dimensionally perceiving the state information of a target area.

[0004] One aspect of the present disclosure provides a method for multi-dimensionally perceiving the state information of a target area. The target area includes a plurality of target points. The method is applied to a server and includes: in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determining target image feature points based on the target image, where the target image feature points are local points with a preset geometric shape in the target image; based on the geographical location information, the orientation information, and the target image feature points, searching for historical scene information of the target point from the database of the server; rendering the historical scene information of the target point to generate a historical image of the target point, and sending the historical image to the terminal; using a preset feature matching algorithm to obtain visual difference information of the target point based on the historical image and the target image, where the visual difference information represents the difference degree between the historical scene information and the target image; generating target state information based on the plurality of visual difference information and sensory information of the plurality of target points, where the target state information indicates the development state of the target area, and the sensory information is determined by the user from candidate sensory information in combination with the display screen of the terminal and the environmental state of the target area, and the historical image and the target image are located on different layers of the display screen.

[0005] According to an embodiment of the present disclosure, the historical scene information is obtained by the following method:

[0006] In response to receiving multiple frames of images containing the target point sent by the above terminal, multiple image feature points are obtained based on the multiple frames of images. The multiple frames of images are collected by the terminal at multiple acquisition directions for the target point; the multiple image feature points are matched and fused to obtain a three-dimensional point cloud model of the target point. The three-dimensional point cloud model represents the structural features of the object at the target point in three-dimensional space; the historical scene information is obtained based on the multiple frames of images and the three-dimensional point cloud model.

[0007] According to an embodiment of the present disclosure, the target point includes a garbage gathering point, and the method further includes: in response to receiving at least one frame of image of the garbage gathering point sent by the terminal, inputting the at least one frame of image into a preset image recognition model to obtain a recognition result of the garbage gathering point. The recognition result is used to generate the target state information.

[0008] According to an embodiment of the present disclosure, the method further includes: in response to receiving multiple candidate target points and the geographical location information of the user sent by the terminal, using a preset path planning algorithm to generate a navigation route based on the multiple candidate target points and the geographical location of the user, and sending the navigation route to the terminal.

[0009] According to an embodiment of the present disclosure, the method further includes: querying restaurant data around each candidate target point based on the multiple candidate target points. The restaurant data includes restaurant types, food types, and food prices; determining the food richness information of each candidate target point based on the restaurant types, the food types, and the food prices. The food richness information is used to generate the target state information.

[0010] According to an embodiment of the present disclosure, the terminal and the server also store a preset encryption key. Sending the navigation route to the terminal includes: encrypting the navigation route based on the preset encryption key to obtain encrypted transmission data; sending the encrypted transmission data to the terminal so that the terminal decrypts the encrypted transmission data based on the preset encryption key to obtain the navigation route.

[0011] Another aspect of the present disclosure provides an apparatus for multi-dimensionally perceiving the status information of a target area, where the target area includes multiple target points, and the apparatus is assembled on a server side and includes: a determination module, configured to, in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determine target image feature points based on the target image, where the target image feature points are local points with a preset geometric shape in the target image; a search module, configured to search for historical scene information of the target point from the database of the server side based on the geographical location information, the orientation information, and the target image feature points; a rendering module, configured to render the historical scene information of the target point to generate a historical image of the target point, and send the historical image to the terminal; a difference calculation module, configured to use a preset feature matching algorithm to obtain visual difference information of the target point based on the historical image and the target image, where the visual difference information represents the difference degree between the historical scene information and the target image; a generation module, configured to generate target status information based on the multiple visual difference information and sensory information of the multiple target points, where the target status information indicates the development status of the target area, and the sensory information is determined by the user from candidate sensory information in combination with the display screen of the terminal and the environmental status of the target area, and the historical image and the target image are located on different layers of the display screen.

[0012] Another aspect of the present disclosure provides an electronic device, including:

[0013] One or more processors;

[0014] A memory for storing one or more programs,

[0015] wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described above.

[0016] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, where the instructions are used to implement the method as described above when executed.

[0017] Another aspect of the present disclosure provides a computer program product, where the computer program product includes computer-executable instructions, and the instructions are used to implement the method as described above when executed.

[0018] According to an embodiment of the present disclosure, by receiving the geographical location information, orientation information, and target image of a target point from a terminal, the target image feature points in the target image are determined, and based on the above geographical location information, orientation information, and target image feature points, historical scene information is found in a database, and the historical image of the target point is rendered therefrom, and the historical image is sent to the terminal to obtain the sensory information of the user, and the target state information capable of characterizing the development state of the target area is generated by combining the sensory information and the visual difference information, so that the user can make an intuitive judgment by comprehensively considering the historical image when generating the target state information, thereby making the sensory information more accurate, and further improving the accuracy of the target state information characterizing the development state of the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0020] Figure 1 Schematically shows an application scenario diagram of a method and device for multi-dimensional perception of the state information of a target area according to an embodiment of the present disclosure;

[0021] Figure 2 Schematically shows a flowchart of a method for multi-dimensional perception of the state information of a target area according to an embodiment of the present disclosure;

[0022] Figure 3 Schematically shows an application architecture diagram of multi-dimensional perception of the state information of a target area according to an embodiment of the present disclosure;

[0023] Figure 4 Schematically shows a structural block diagram of a device for multi-dimensional perception of the state information of a target area according to an embodiment of the present disclosure;

[0024] Figure 5 Schematically shows a block diagram of an electronic device suitable for implementing the method for multi-dimensional perception of the state information of a target area according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0026] The terms used herein are for describing specific embodiments only and are not intended to limit the present disclosure. Terms such as "including" and "comprising" used herein indicate the presence of the described features, steps, operations, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted to have a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0029] In the embodiments of the present disclosure, in aspects such as the collection, update, analysis, processing, use, transmission, provision, disclosure, and storage of the involved data (for example, including but not limited to user personal information), they all comply with the provisions of relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and to safeguard the security of user personal information, network security, and national security.

[0030] In the embodiments of the present disclosure, before obtaining or collecting user personal information, the authorization or consent of the user is obtained.

[0031] With the rapid development of global urbanization, the urban population has increased sharply and the scale has been continuously expanding. The urbanization rate has reached a relatively high level, and many cities are facing pressures in multiple aspects such as population, resources, and the environment. The aging of urban infrastructure and ecological environment problems are becoming increasingly severe, and the layout and supply of public service facilities are difficult to meet the needs of residents. In order to comprehensively understand the development status of a city, it is necessary to conduct a physical examination of the city. However, the existing methods for urban physical examination require collecting data at target points in the target urban area and manually judging and scoring the currently collected data without historical data comparison, thus unable to accurately determine the evaluation score corresponding to the current state of the target points, resulting in inaccurate urban physical examination results.

[0032] In view of this, embodiments of the present disclosure provide a method for multi-dimensionally perceiving the status information of a target area, where the target area includes multiple target points, and the method is applied to a server. The method includes: in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determining target image feature points based on the target image, where the target image feature points are local points with a preset geometric shape in the target image; based on the geographical location information, the orientation information, and the target image feature points, looking up the historical scene information of the target point from the database of the server; rendering the historical scene information of the target point to generate a historical image of the target point, and sending the historical image to the terminal; using a preset feature matching algorithm to obtain visual difference information between the historical image and the target image based on the historical image and the target image, where the visual difference information characterizes the difference degree between the historical scene information and the target image; generating target status information based on the multiple visual difference information and sensory information of the multiple target points, where the target status information indicates the development status of the target area, and the sensory information is determined by the user from the candidate sensory information in combination with the display screen of the terminal and the environmental status of the target area, and the historical image and the target image are located on different layers of the display screen.

[0033] Figure 1 Schematically shows an application scenario diagram of a method and device for multi-dimensionally perceiving the status information of a target area according to an embodiment of the present disclosure.

[0034] As Figure 1 shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0035] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and mobile GIS (Geographic Information System) devices, etc.

[0037] The server 105 may be a server that provides various services. For example, it may be a background management server (only for illustration) that supports the websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server can analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0038] It should be noted that the method for multi-dimensionally perceiving the status information of the target area provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the device for multi-dimensionally perceiving the status information of the target area provided by the embodiments of the present disclosure can generally be set in the server 105. The method for multi-dimensionally perceiving the status information of the target area provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the device for multi-dimensionally perceiving the status information of the target area provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0039] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0040] Figure 2 are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0041] As Figure 2 shown, the method includes operations S210 to S250.

[0042] According to the embodiments of the present disclosure, the above-mentioned target area includes multiple target points, and the above method can be applied to the server side.

[0043] In operation S210, in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determine the target image feature points based on the target image.

[0044] Among them, the target image feature points are local points with a preset geometric shape in the target image.

[0045] According to the embodiments of the present disclosure, before the start of the acquisition work, it is necessary to conduct preliminary research and analysis on the target city, collect relevant materials such as the geographical information, population distribution, functional zoning, and economic development of the city, and further analyze the characteristics and development needs of the city. Then, select key areas and densely populated areas in the city as target points. Exemplarily, for a city with tourism resources as the core, it is necessary to focus on tourist attractions and surrounding areas. Therefore, multiple scenic spots in the city can be selected as target points. Outside the tourism core area, the surrounding areas of the main transportation hubs in the city can also be selected as target points, such as near high-speed railway stations and central business districts. Further, for the specific selection of target points, representative points need to be selected according to different functions and characteristics. For example, in a tourist attraction, select the tourist service center, main scenic spot entrances, popular tour routes, etc. as target points to understand the experience of tourists during the tourism process. In the commercial area, points such as the entrances and internal passages of large shopping malls, supermarkets, and commercial streets can be selected to consider the impact of the commercial environment on consumers; in the historical and cultural block, ancient monument scenic spots, areas and points with concentrated folk houses can be selected to evaluate the impact of historical and cultural protection and utilization on residents and tourists. The selection of each target point should be able to fully reflect the urban physical examination index situation of the area where it is located to ensure that the collected data is representative and comprehensive.

[0046] In operation S220, based on the geographical location information, orientation information, and target image feature points, search for the historical scene information of the target point from the database of the server.

[0047] According to the embodiments of the present disclosure, the geographical location information is the precise coordinates of the target point, usually obtained through the Global Positioning System or other positioning technologies. The orientation information refers to the orientation angle of the terminal device, usually obtained through sensors of the terminal (such as gyroscopes, accelerometers, magnetometers, etc.). The orientation information is used to determine the current line of sight direction of the user so that the system can provide enhanced content consistent with the user's field of view direction. The target image feature points refer to the key points in the image captured by the user device, which are extracted through image recognition technology. The feature points are used to identify the objects or scenes in the user's current field of view and match them with the historical scene information in the database.

[0048] In operation S230, historical scene information of the target point is rendered to generate a historical image of the target point, and the historical image is sent to the terminal.

[0049] According to an embodiment of the present disclosure, historical images of target points can be generated through rendering. For example, when displaying historical buildings, images of the building in different historical periods can be generated. After rendering, the images can be sent to users so that the users can view these images on terminal devices and compare them with real scenes.

[0050] In operation S240, a preset feature matching algorithm is used to obtain visual difference information of the target point based on the historical image and the target image, where the visual difference information represents the difference between the historical scene information and the target image.

[0051] According to an embodiment of the present disclosure, the above-mentioned preset feature matching algorithm may be, for example, a scale-invariant feature transformation algorithm or an accelerated robust feature algorithm, and the present disclosure does not impose any limitation on this.

[0052] In operation S250, target state information is generated based on the plurality of visual difference information and sensory information of the plurality of target points.

[0053] Among them, the target state information indicates the development state of the target area, the sensory information is determined by the user from the candidate sensory information in combination with the display screen of the terminal and the environmental state of the target area, and the historical image and the target image are located in different layers of the display screen.

[0054] According to the disclosed embodiment, the candidate sensory information is determined based on landscape ecology, which focuses on human perception and experience of the ecosystem, emphasizes the harmonious coexistence of man and nature, and focuses on human vision, hearing, smell, taste and touch for the external environment. Therefore, the candidate sensory information may include, for example, "facility operability", "facility integrity", "richness of children's parent-child stores", "richness of cultural and sports entertainment stores", "orderliness", "cleanliness", "urban physical examination resident satisfaction: visual", "richness of catering stores" and other information. Furthermore, the characteristic indicators of urban physical examination (i.e., the candidate sensory information) are based on urban residents and the five senses of "sight, hearing, smell, taste and touch" of urban residents are the research objects. Therefore, the types of candidate sensory information can be determined around the integration of urban buildings and natural landscapes in vision, the balance of urban environment and artificial sounds in hearing, the fresh air and reasonable distribution of odors in smell, the characteristic food culture and high-quality drinking water in taste, and the impact of the touch of public facilities and pleasant microclimate on the human body.

[0055] According to an embodiment of the present disclosure, the above sensory information may include, for example, the ambient sound decibel level and odor situation at the target location. The aforementioned ambient sound decibel level can be obtained by relevant staff at the target location based on the built-in sensors of the terminal. The terminal device can be connected to a sound level meter and a portable multi-gas detection sensor. After the relevant staff arrives at the target location, the application can be used to determine whether the current location is consistent with the collection location required by the task, and ensure that the device and the built-in sensors are functioning properly. The terminal parameters, such as the resolution of the camera module, image brightness, and positioning navigation module accuracy, are adjusted and confirmed to ensure the accuracy of the urban environment information of the mobile phone, so as to accurately reflect the actual situation of the street. At the same time, by checking the task list in the application, the types and quantities of information to be collected are clarified. The information collected may include, for example, a panoramic view of the street where the target location is located, the street environment decibel level, and information on the detailed evaluation of environmental hygiene, etc.

[0056] According to an embodiment of the present disclosure, by receiving the geographical location information, orientation information, and target image of the target location sent by the terminal, the target image feature points in the target image are determined, and based on the above geographical location information, orientation information, and target image feature points, historical scene information is found in the database, and it is rendered to obtain the historical image of the target location. The historical image is sent to the terminal to obtain the user's sensory information, and the target state information capable of characterizing the development state of the target area is generated by combining the sensory information and the visual difference information, so that when generating the target state information, the user can comprehensively use the historical image for intuitive judgment, thereby making the sensory information more accurate and further improving the accuracy of the target state information characterizing the development state of the target area.

[0057] According to an embodiment of the present disclosure, the historical scene information is obtained by the following method: in response to receiving multiple frames of images containing the target location sent by the terminal, multiple image feature points are obtained based on the multiple frames of images. The multiple frames of images are collected by the terminal at the target location in multiple acquisition directions; the multiple image feature points are matched and fused to obtain a three-dimensional point cloud model of the target location. The three-dimensional point cloud model characterizes the structural features of the objects at the target location in three-dimensional space; the historical scene information is obtained based on the multiple frames of images and the three-dimensional point cloud model.

[0058] According to the disclosed embodiment, when relevant personnel collect data of target points, they can shoot the target points from multiple collection directions to obtain multiple frames of images, which capture the appearance and structural information of the target points from different angles. When collecting urban environmental information, relevant staff can stand on, for example, a commanding point on the street or an area with a wide field of view and no obstacles, and shoot target images that cover the overall street environment, buildings, roads, greenery and other elements. When shooting, attention should be paid to the integrity and symmetry of the picture to avoid occlusion or deformation. The application on the terminal will automatically record the collection time and geographic location information and associate it with the collection task, and transmit the target image to the server, and other authorized users can view it in real time through the application. More specifically, when collecting close-up photos of stores, it is necessary to shoot details such as signs, facade decoration, and store layout from multiple angles and directions to highlight the store characteristics and the store's operating status.

[0059] According to the disclosed embodiment, feature points in multiple frames of images can be matched to find corresponding feature points in different images, and then the matched feature points can be fused using a 3D reconstruction algorithm (such as a structural optics algorithm, a stereo vision algorithm, etc.) to generate a 3D point cloud model. Furthermore, data such as materials can be added to the 3D point cloud model to generate historical scene information.

[0060] According to the embodiments of the present disclosure, by collecting images of the target points and performing matching and fusion based on the image feature points in the images, a three-dimensional point cloud model is generated, and historical scene information is further generated based on multiple frames of images and the three-dimensional point cloud model. The target points can be modeled more conveniently, providing a comparison basis for the subsequent determination of the target area status information, thereby improving the accuracy of the target area status information.

[0061] According to an embodiment of the present disclosure, the target point includes a garbage collection point, and the method further includes: in response to receiving at least one frame of image of the garbage collection point sent by the terminal, inputting the at least one frame of image into a preset image recognition model to obtain a recognition result of the garbage collection point, and the recognition result is used to generate the target status information.

[0062] According to the disclosed embodiment, the type and amount of garbage at the garbage gathering point can be identified through images, and the odor situation at the target point can be determined through the terminal external device. The situation of the target point can also be recorded in detail through text description, taking photos or recording videos. During the collection process, the application uploads information in real time, and the application end updates the environmental sanitation situation in real time.

[0063] According to an embodiment of the present disclosure, the application will also perform real-time analysis on the collected data. When it is found that the amount of garbage accumulation has increased significantly or the number of damaged public facilities has increased abnormally, etc., the application end will issue a warning to remind relevant staff to handle it in time, so as to ensure that the urban environment maintains a good state.

[0064] According to an embodiment of the present disclosure, the method further includes: in response to receiving a plurality of candidate target points and the geographical location information of the user sent by the terminal, using a preset path planning algorithm, generating a navigation route based on the plurality of candidate target points and the geographical location of the user, and sending the navigation route to the terminal.

[0065] According to an embodiment of the present disclosure, before the collector goes to the target point, it is necessary to ensure that the terminal has sufficient power, and open the corresponding application developed based on the Android platform in advance. On the application interface, enter the personal account and password to log in, and the system automatically verifies the identity information to ensure that relevant staff have the corresponding operation permissions. After successful login, the application will automatically connect to the server to obtain the latest collection tasks and relevant data dictionary information of the target city, including the indicators to be concerned about in this collection, the number of collection indicators, etc.

[0066] According to an embodiment of the present disclosure, the application will automatically use the navigation module of the terminal to determine the current position of the user (i.e., relevant staff) and display it on the map. New users can select the target point, and the application will plan the best navigation route to reach the target point. When approaching the target point, the application will issue a reminder to inform the collector that the target point is about to be reached.

[0067] According to an embodiment of the present disclosure, the above preset path planning algorithm can be algorithms such as a random search space algorithm, an optimization algorithm for simulating group behavior, and an ant colony algorithm. The present disclosure does not limit this.

[0068] According to an embodiment of the present disclosure, through the built-in navigation function of the application, a navigation route can be generated based on the geographical location information of the user after the user selects the candidate target point, and then the user can be conveniently navigated to the target point for data collection.

[0069] Figure 3 Schematically shows an application architecture diagram of the state information of the multi-dimensional perception target area according to an embodiment of the present disclosure.

[0070] Such as Figure 3As shown, the architecture includes a list selection module, a city map management module, a data dictionary module, a city physical examination index information collection module, and a data transmission management module. Among them, the list selection module is used for classified management of target cities; the city map management module can integrate basic geographic data such as urban topography, road networks, and buildings, as well as data related to city physical examinations, and perform visual display for easy data location, query, and analysis; the data dictionary module stores data definitions, classifications, and specifications for city physical examination designs determined based on landscape ecology, facilitating the clarification of data meanings, types, and value ranges; the city physical examination index information collection module can collect data closely related to human senses through sensors built into the terminal or external devices; the data transmission management module is used for data upload and download, and through network transmission technology and encryption technology, securely and quickly transmits the collected data to the server database for storage and management.

[0071] According to an embodiment of the present disclosure, the application also supports intuitively viewing the status information of a certain area, such as the satisfaction scores of different streets, the distribution of store richness, etc. Through operations such as zooming in and out and panning of the map, the spatial characteristics and development differences of the city can be deeply understood.

[0072] According to an embodiment of the present disclosure, the above data dictionary may include street environment satisfaction indicators, which cover two keyword fields of "cleanliness" and "orderliness". "Cleanliness" is used to describe the sanitary cleanliness of the street, including aspects such as whether the road surface is clean and whether there is garbage accumulation; "orderliness" focuses on the order status of the street, such as whether the stalls are neatly arranged and whether the vehicle parking is regulated. The street environment richness indicator is further divided into fields such as "richness of catering stores", "richness of cultural, sports and entertainment stores", "richness of life service stores", "richness of retail stores", "richness of children and parent-related stores", etc. "Richness of catering stores" reflects the quantity, variety, and distribution of various catering stores on the street; "richness of cultural, sports and entertainment stores" reflects the diversity of cultural, sports, and entertainment consumption places provided by the street; "richness of life service stores" covers stores closely related to residents' daily lives, such as barbershops, laundries, repair shops, etc.; "richness of retail stores" represents the richness of various retail stores; "richness of children and parent-related stores" focuses on stores related to children, such as children's toy stores and parent-child amusement parks, and their distribution and quantity. In addition to the above indicators, the data dictionary also comprehensively defines other indicators, aiming at realizing the systematic perception of the public, and constructing indicator fields including but not limited to aspects such as ecological environment, traffic conditions, and public service facilities.

[0073] According to an embodiment of the present disclosure, the above method further includes querying restaurant data around each candidate target location based on the multiple candidate target locations, where the restaurant data includes restaurant types, food types, and food prices; determining food richness information for each candidate target location based on the restaurant types, the food types, and the food prices, and the food richness information is used to generate the target status information.

[0074] According to an embodiment of the present disclosure, the above restaurant data can be obtained through a food service platform or a standardized food interface, and the food richness information is integrated into the target status information for evaluating the overall development status of the target area. For example, in urban planning, the food richness information can reflect the living convenience and attractiveness of an area.

[0075] According to an embodiment of the present disclosure, the terminal and the server further store a preset encryption key, and sending the navigation route to the terminal includes: encrypting the navigation route based on the preset encryption key to obtain encrypted transmission data; sending the encrypted transmission data to the terminal so that the terminal decrypts the encrypted transmission data based on the preset encryption key to obtain the navigation route.

[0076] According to an embodiment of the present disclosure, the above preset encryption key is a key pre-shared between the terminal and the server for encrypting and decrypting data. This key can be a symmetric key. When encrypting the navigation route, symmetric encryption can be performed. The encrypted data is unreadable during transmission, and only the terminal with the correct key can decrypt it. The encrypted navigation route data is sent to the terminal. Since the data is encrypted, even if it is intercepted during transmission, the attacker cannot read its content, thus protecting the confidentiality and integrity of the data.

[0077] According to an embodiment of the present disclosure, encrypting the navigation route through the encryption key to obtain encrypted transmission data and transmitting the encrypted transmission data to the terminal, and the terminal decrypts it based on the preset encryption key, can effectively prevent the navigation route data from being stolen or tampered with during transmission, thereby protecting the privacy and data security of users.

[0078] According to an embodiment of the present disclosure, the evaluation form of the above sensory information can be, for example, score data. Relevant staff or application users can evaluate the current target location based on historical model data on the corresponding application interface according to their own experience. The application provides a sliding bar or a digital input box with scores from 1 to 10, and users can select the corresponding score according to their judgment or give a negative score. Exemplarily, if a user believes that the "cleanliness" of the street can be scored 7 points, then the user can select 7 points in the scoring area corresponding to the "cleanliness" indicator on the application interface.

[0079] According to the disclosed embodiment, after the relevant staff has completed the data collection, nearby residents can also evaluate and score the historical data of the target point, such as the green coverage, vegetation type distribution, etc. When the relevant staff reviews the relevant scores of street greening, they use enhanced display technology to superimpose the historical data of green coverage, vegetation type distribution, etc. in different time periods on the real scene in the form of virtual images, and compare them with the descriptions and scores in the feedback from nearby residents. If it is found that residents give a low score to the greening of a certain street, but the historical scene data displayed by the virtual image shows that the greening of the street is in good condition and has not changed significantly recently, the relevant staff can determine that there is an abnormality in the scoring this time. For relevant evaluation indicators such as air quality and noise level, the data collector can aim the terminal camera at the corresponding monitoring point, and the application interface will present the historical data curve of the monitoring point and the real-time monitoring data distribution of the surrounding area in real time. If it is found that residents report that the noise data in a certain area exceeds the standard and give a low score, but the historical noise data displayed in the area is within the normal range, and the real-time data of other surrounding monitoring points are also normal, the relevant staff can determine that there is an unreasonable place in the score, and will contact the relevant user to ask about the specific basis of the score and the actual feeling at the time, and understand the specific reasons behind it. After completing the communication, the relevant staff will revise and review the score, and finally compare the new data with historical data, surrounding area data and field scenes to ensure the accuracy of the data. After the review is passed, the relevant staff confirms the user's scoring results in the application and uploads them to the urban physical examination database to provide reliable technical support for subsequent urban planning, management and decision-making.

[0080] According to the disclosed embodiment, the application also provides a scoring explanation and feedback function. After scoring, the user can fill in a brief text description in the application and upload the reason and basis for the scoring. For example, the user gave a score of 5 to the "orderliness" indicator, and can explain in the scoring description that "there are many vehicles parked randomly, affecting the traffic order."

[0081] According to the embodiments of the present disclosure, relevant staff can introduce the meanings and scoring criteria of the evaluation indicators of the target points to residents through the application interface. Taking "cleanliness" as an example, relevant staff will explain this evaluation indicator, which mainly focuses on the sanitary cleaning degree of the street, specifically including aspects such as whether the road surface is clean, whether there is garbage accumulation, and whether public facilities are tidy. At the same time, the specific reference for scoring will be clearly informed to users: 1 point represents very dissatisfied. For example, if a user is at the collection point and sees that the street is extremely dirty and messy, with garbage discarded randomly and visible everywhere, and the surfaces of public facilities are covered with dirt and severely damaged, a 1-point evaluation can be given; while 10 points indicate very satisfied. If the street seen by the user is spotless, the road surface is as clean as if washed, without any garbage residue, and the public facilities are not only clean and tidy but also well maintained, a 10-point score can be given. For the "richness of catering stores" indicator, relevant staff will also explain it in detail. This indicator is used to measure the number, types of catering stores on the street, and whether they can meet the needs of different people. In the actual scoring scenario, if a resident is around the target point and finds that there are hardly any catering stores on the street, only one or two can be found, and the available dishes are extremely limited and cannot meet the basic dietary needs, the user can give a 1-point evaluation based on this environmental information; on the contrary, if the resident sees row upon row of catering stores, covering various types such as Chinese cuisine, Western cuisine, fast food, and special snacks, ranging from affordable snack bars to high-end restaurants, and can fully meet the needs of various taste preferences and consumption levels, then the user can give a 10-point satisfactory evaluation according to the actual situation of the current collection point.

[0082] Based on the above method for multi-dimensionally perceiving the state information of the target area, the present disclosure also provides a device for multi-dimensionally perceiving the state information of the target area. The following will be combined with Figure 4 to describe this device in detail.

[0083] Figure 4 The structural block diagram of the device for multi-dimensionally perceiving the state information of the target area according to the embodiments of the present disclosure is schematically shown.

[0084] As Figure 4 shown, the device 400 for multi-dimensionally perceiving the state information of the target area in this embodiment includes a determination module 410, a search module 420, a rendering module 430, a difference calculation module 440, and a generation module 450.

[0085] The determination module 410 is configured to, in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determine the target image feature points based on the target image, where the target image feature points are local points with a preset geometric shape in the target image. In one embodiment, the determination module 410 can be used to perform the operation S210 described above, which will not be elaborated here.

[0086] The search module 420 is configured to search for historical scene information of a target location from a database of a server based on geographical location information, orientation information, and target image feature points. In one embodiment, the search module 420 may be configured to perform the operation S220 described above, which will not be elaborated herein.

[0087] The rendering module 430 is configured to render the historical scene information of the target location to generate a historical image of the target location and send the historical image to the terminal. In one embodiment, the rendering module 430 may be configured to perform the operation S230 described above, which will not be elaborated herein.

[0088] The difference calculation module 440 is configured to obtain visual difference information of the target location based on the historical image and the target image by using a preset feature matching algorithm. The visual difference information represents the degree of difference between the historical scene information and the target image. In one embodiment, the difference calculation module 440 may be configured to perform the operation S240 described above, which will not be elaborated herein.

[0089] The generation module 450 is configured to generate target status information based on multiple visual difference information and sensory information of multiple target locations. The target status information indicates the development status of the target area. The sensory information is determined by the user from candidate sensory information in combination with the display screen of the terminal and the environmental status of the target area. The historical image and the target image are located on different layers of the display screen. In one embodiment, the generation module 450 may be configured to perform the operation S250 described above, which will not be elaborated herein.

[0090] According to embodiments of the present disclosure, any multiple of the determination module 410, the search module 420, the rendering module 430, the difference calculation module 440, and the generation module 450 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the determination module 410, the search module 420, the rendering module 430, the difference calculation module 440, and the generation module 450 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the determination module 410, the search module 420, the rendering module 430, the difference calculation module 440, and the generation module 450 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0091] Figure 5 A block diagram of an electronic device suitable for implementing a method for state information of a multi-dimensional perception target area according to an embodiment of the present disclosure is schematically shown.

[0092] As Figure 5 shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage section 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 501 may also include on-board memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of the present disclosure.

[0093] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. It should be noted that the programs may also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0094] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, and the input / output (I / O) interface 505 is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read from it can be installed into the storage portion 508 as needed.

[0095] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0096] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503.

[0097] An embodiment of the present disclosure further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the method for obtaining the state information of the multi-dimensional perception target area provided by the embodiment of the present disclosure.

[0098] When the computer program is executed by the processor 501, it executes the above functions defined in the system / apparatus of the embodiment of the present disclosure. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0099] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 509, and / or be installed from the removable medium 511. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0100] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or be installed from the removable medium 511. When the computer program is executed by the processor 501, it executes the above functions defined in the system of the embodiment of the present disclosure. According to an embodiment of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0101] According to embodiments of the present disclosure, program code for executing the computer programs provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages and / or assembly / machine languages. The programming languages include, but are not limited to, programming languages such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0103] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0104] The above describes the embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A method for the state information of a multi-dimensional perception target area, characterized in that, The target area includes multiple target points, and the method is applied to the server. The method includes: In response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determining target image feature points based on the target image. The target image feature points are local points with a preset geometric shape in the target image; Based on the geographical location information, the orientation information, and the target image feature points, searching for the historical scene information of the target point from the database of the server; Rendering the historical scene information of the target point to generate a historical image of the target point, and sending the historical image to the terminal; Using a preset feature matching algorithm, obtaining visual difference information of the target point based on the historical image and the target image. The visual difference information characterizes the difference degree between the historical scene information and the target image; Generating target state information based on the multiple visual difference information and sensory information of the multiple target points. The target state information indicates the development state of the target area. The sensory information is determined by the user from the candidate sensory information in combination with the display screen of the terminal and the environmental state of the target area. The historical image and the target image are located on different layers of the display screen.

2. The method according to claim 1, wherein The historical scene information is obtained through the following method: In response to receiving multiple frames of images sent by the terminal and including the target point, obtaining multiple image feature points based on the multiple frames of images. The multiple frames of images are collected by the terminal for the target point in multiple acquisition directions; Performing matching and fusion on the multiple image feature points to obtain a three-dimensional point cloud model of the target point. The three-dimensional point cloud model characterizes the structural features of the object at the target point in three-dimensional space; Obtaining the historical scene information based on the multiple frames of images and the three-dimensional point cloud model.

3. The method according to claim 1, characterized in that, The target point includes a garbage collection point, and the method further includes: In response to receiving at least one frame of image of the garbage collection point sent by the terminal, inputting the at least one frame of image into a preset image recognition model to obtain an identification result of the garbage collection point. The identification result is used to generate the target state information.

4. The method according to claim 1, wherein The method further includes: In response to receiving multiple candidate target points and the geographical location information of the user sent by the terminal, using a preset path planning algorithm to generate a navigation route based on the multiple candidate target points and the geographical location of the user, and sending the navigation route to the terminal; 5. The method according to claim 4, characterized in that, The method further includes: Querying for restaurant data around each candidate target point based on the multiple candidate target points. The restaurant data includes restaurant types, food types, and food prices; Determining food richness information of each candidate target point based on the restaurant types, the food types, and the food prices. The food richness information is used to generate the target state information.

6. The method according to claim 4, characterized in that, The terminal and the server also store a preset encryption key. Sending the navigation route to the terminal includes: Encrypt the navigation route based on the preset encryption key to obtain encrypted transmission data; Send the encrypted transmission data to the terminal so that the terminal decrypts the encrypted transmission data based on the preset encryption key to obtain the navigation route.

7. A device for multi-dimensional perception of the state information of a target area, characterized in that, The target area includes multiple target points. The device is assembled on the server side. The device includes: A determination module, configured to, in response to receiving the geographical location information of the target point, the orientation information of the terminal relative to the target point, and the target image collected by the terminal for the target point, determine target image feature points based on the target image, where the target image feature points are local points with a preset geometric shape in the target image; A search module, configured to search for historical scene information of the target point from the database of the server based on the geographical location information, the orientation information, and the target image feature points; A rendering module, configured to render the historical scene information of the target point to generate a historical image of the target point, and send the historical image to the terminal; A difference calculation module, configured to use a preset feature matching algorithm to obtain visual difference information of the target point based on the historical image and the target image, where the visual difference information characterizes the difference degree between the historical scene information and the target image; A generation module, configured to generate target state information based on the multiple visual difference information and sensory information of the multiple target points, where the target state information indicates the development state of the target area, and the sensory information is determined by the user from candidate sensory information in combination with the display screen of the terminal and the environmental state of the target area, and the historical image and the target image are located on different layers of the display screen.

8. An electronic device, comprising: One or more processors; A memory, configured to store one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images