Park monitoring method and park monitoring device

Abstract

This disclosure relates to a park monitoring method, including: determining a first path from a first object to a first location; generating a 3D virtual image of the first object moving along the first path based on the perspective of the first object; and displaying a real-world image of the first location when the first object is at the first location. According to embodiments of this disclosure, a 3D virtual image of the first object moving along the first path can be displayed based on the perspective of the first object, and a real-world image of the first location can be displayed when the first object is at the first location. This process combines the 3D virtual image and the displayed image, facilitating monitoring personnel to intuitively understand the situation along the first path and to intuitively and accurately observe the behavior of the first object at the first location.

Description

Technical Field

[0001] This disclosure relates to the field of display technology, and in particular to park monitoring methods, park monitoring devices, electronic devices, and computer-readable storage media. Background Technology

[0002] In related technologies, the scene is displayed either through 3D virtual images or through real-world images, but the combination of 3D virtual images and real-world images is not reasonable, making it difficult to achieve good display effects in certain application scenarios. Summary of the Invention

[0003] This disclosure provides a park monitoring method, park monitoring device, electronic device, and computer-readable storage medium to address the shortcomings of related technologies.

[0004] According to a first aspect of the present disclosure, a park monitoring method is provided, comprising:

[0005] Determine the first path from the first object to the first position;

[0006] Generate a 3D virtual image of the first object moving along the first path based on the perspective of the first object.

[0007] When the first object is located at the first position, a real-world image of the first position is displayed.

[0008] Optionally, where the first object is the handler of the alarm event, the method further includes, before determining the first path from the first object to the first location:

[0009] When an alarm event is detected, a 3D virtual image of the environment in which the alarm event occurs is displayed, and a marker for the alarm event is displayed in the 3D virtual image;

[0010] Upon receiving the first command, display a 3D virtual image of the first location;

[0011] Upon receiving the second command, the handler and the handler's second location are determined;

[0012] The determination of the first path from the first object to the first position includes:

[0013] Determine the first path from the second location to the first location.

[0014] Optionally, the 3D virtual image displaying the first location includes:

[0015] Determine a second path from the third position at the current monitoring viewpoint to the first position;

[0016] A 3D virtual image moving along the second path is generated based on the current monitoring perspective.

[0017] Optionally, the 3D virtual image displaying the first location includes:

[0018] In the 3D virtual image of the environment where the alarm event is located, the building where the alarm event is located is cut, split, and enlarged to display the 3D virtual image of the first location.

[0019] Optionally, the method further includes:

[0020] As the first position changes over time, the trajectory of the change in the first position is recorded;

[0021] A third position is determined based on the motion trajectory, wherein the third position is a position on the motion trajectory, or a position after the first position has changed;

[0022] The determination of the first path from the first object to the first position includes:

[0023] Determine the first path from the second position to the changed first position.

[0024] Optionally, the method further includes:

[0025] Determine the target perspective based on the motion trajectory of the first position;

[0026] A 3D virtual image moving along the trajectory is generated based on the target viewpoint.

[0027] Optionally, the real-world image displaying the first location includes:

[0028] The real-world image of the first location is fused and displayed in the 3D virtual image of the first location.

[0029] Optionally, the first object is a vehicle, and before determining the first path from the first object to the first location, the method further includes:

[0030] Identify the target parking space and its first location in the parking lot;

[0031] The determination of the first path from the first object to the first position includes:

[0032] Determine the first path from the location of the vehicle to the first location.

[0033] Optionally, the method further includes:

[0034] Record and / or display a second object entering or exiting the vehicle, as well as the trajectory of the second object.

[0035] Optionally, the method further includes:

[0036] Send a 3D virtual image of the first object moving along the first path to the first object.

[0037] Optionally, the method further includes:

[0038] Record a 3D virtual image of the first object moving along the first path, and / or record a real-world image of the first position when the first object is located at the first position.

[0039] According to a second aspect of the present disclosure, a park monitoring device is provided, comprising:

[0040] The path determination module is used to determine the first path from the first object to the first location;

[0041] A virtual image generation module is used to generate a 3D virtual image of the first object moving along the first path based on the perspective of the first object.

[0042] The real-view display module is used to display a real-view image of the first position when the first object is located at the first position.

[0043] Optionally, the first object is the handler for handling alarm events, and the device further includes:

[0044] A virtual image display module is configured to display a 3D virtual image of the environment in which the alarm event occurs when an alarm event is detected, and to display a marker of the alarm event in the 3D virtual image; and to display a 3D virtual image of the first location when a first command is received;

[0045] The first position determination module is used to determine the handler and the second position of the handler when the second command is received;

[0046] The path determination module is used to determine a first path from the second location to the first location.

[0047] Optionally, the virtual image display module is used to determine a second path from the third position of the current monitoring viewpoint to the first position; and to generate a 3D virtual image that moves along the second path based on the current monitoring viewpoint.

[0048] Optionally, the virtual image display module is used to cut, split, and enlarge the building where the alarm event is located in the 3D virtual image of the environment where the alarm event is located, so as to display the 3D virtual image of the first location.

[0049] Optionally, the device further includes:

[0050] The trajectory recording module is used to record the motion trajectory of the first position as it changes over time.

[0051] A position prediction module is used to determine a third position based on the motion trajectory, wherein the third position is a position on the motion trajectory, or a position after the first position has changed;

[0052] The path determination module is used to determine a first path from the second position to the changed first position.

[0053] Optionally, the device further includes:

[0054] A virtual image generation module is used to determine the target viewpoint based on the motion trajectory of the first position; and to generate a 3D virtual image that moves along the trajectory based on the target viewpoint.

[0055] Optionally, the real-scene park monitoring module is used to fuse and display the real-scene image of the first location into the 3D virtual image of the first location.

[0056] Optionally, the first object is a vehicle, and the device further includes:

[0057] The second location determination module is used to determine the target parking space and the first location of the target parking space in the parking lot;

[0058] The path determination module is used to determine a first path from the location of the vehicle to the first location.

[0059] Optionally, the device further includes:

[0060] A recording and display module is used to record and / or display a second object entering or exiting the vehicle, as well as the movement trajectory of the second object.

[0061] Optionally, the device further includes:

[0062] An image sending module is used to send a 3D virtual image of the first object moving along the first path to the first object.

[0063] Optionally, the device further includes:

[0064] The image recording module is used to record a 3D virtual image of the first object moving along the first path, and / or record a real-world image of the first position when the first object is located at the first position.

[0065] According to a third aspect of the present disclosure, a park monitoring system is provided, comprising:

[0066] processor;

[0067] Memory used to store processor-executable instructions;

[0068] The processor is configured to implement the above method.

[0069] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method.

[0070] According to the above embodiments, a 3D virtual image of the first object moving along the first path can be displayed from the perspective of the first object, and a real-world image of the first position can be displayed when the first object is in the first position. In this process, the 3D virtual image and the displayed image are combined to facilitate the monitoring personnel to intuitively understand the situation of the first path and to intuitively and accurately observe the behavior of the first object in the first position.

[0071] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 This is a schematic flowchart illustrating a park monitoring method according to an embodiment of the present disclosure.

[0074] Figure 2 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0075] Figure 3 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0076] Figure 4 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0077] Figure 5 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0078] Figure 6This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0079] Figures 7A to 7M This is a schematic diagram illustrating an application scenario according to an embodiment of the present disclosure.

[0080] Figure 8 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0081] Figure 9 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0082] Figures 10A to 10G This is a schematic diagram illustrating another application scenario according to an embodiment of the present disclosure.

[0083] Figure 11 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0084] Figure 12 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of the present disclosure.

[0085] Figure 13 This is a schematic block diagram of a park monitoring device according to an embodiment of the present disclosure.

[0086] Figure 14 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure.

[0087] Figure 15 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure.

[0088] Figure 16 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure.

[0089] Figure 17 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure.

[0090] Figure 18 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure.

[0091] Figure 19 This is a schematic block diagram of another park monitoring device according to an embodiment of the present disclosure. Detailed Implementation

[0092] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0093] Figure 1 This is a schematic flowchart illustrating a park monitoring method according to an embodiment of the present disclosure. The park monitoring method is applicable to park monitoring systems, which can be applied to scenarios such as buildings, parking lots, and office parks. The park monitoring system may include multiple image acquisition devices, such as cameras and video recorders. These image acquisition devices are positioned at different locations within the scene and can acquire images from various locations within the scene. The acquired images are then sent to the processor of the park monitoring system, which can process the received images, for example, converting them into 3D virtual images.

[0094] like Figure 1 As shown, the park monitoring method may include the following steps:

[0095] In step S101, a first path from the first object to the first position is determined;

[0096] In step S102, a 3D virtual image of the first object moving along the first path is generated based on the perspective of the first object.

[0097] In step S103, when the first object is located at the first position, a real-world image of the first position is displayed.

[0098] In one embodiment, the first object can be a person or a thing, depending on the application scenario. When it is determined that the first object needs to move from its current position to the first position, a path from the first object to the first position can be determined. The algorithm for planning the first path can be selected as needed; for example, the NavMesh algorithm can be used to plan the first path.

[0099] In one embodiment, the viewing angle of the first object can be determined based on the direction of movement of the first object. For example, the viewing angle of the first object can be directed towards a direction lower than the direction of movement of the object, and the viewing angle is located at a preset height. For example, if the first object is a person, the preset height can be the eye height of an average adult, such as 1.65 meters. If the first object is a vehicle, the preset height can be the eye height of the vehicle driver.

[0100] In this process, an image acquisition device near the first path can be used to acquire environmental images of the real-time location. Then, the acquired environmental images can be converted into 3D virtual images according to the perspective of the first object, thereby generating a 3D virtual image observed from the perspective of the first object. Furthermore, based on the movement of the first object along the first path, multiple frames of 3D virtual images can be continuously generated. For example, one frame of 3D virtual image can be generated every preset time. Multiple consecutive frames of 3D virtual images can constitute a 3D video. The 3D video simulates the 3D virtual environment observed from the perspective of the first object as it moves along the first path.

[0101] In one embodiment, a 3D video composed of continuous 3D virtual images can be viewed by monitoring personnel to intuitively understand the situation of the first path, or it can be viewed by the first object, such as a vehicle, in which case the 3D video can be sent to the driver of the vehicle to provide the driver with intuitive navigation.

[0102] Furthermore, when the first object is in the first position, a real-world image of the first position can be displayed so that monitoring personnel can intuitively and accurately observe the behavior of the first object in the first position.

[0103] According to embodiments of this disclosure, a 3D virtual image of the first object moving along a first path can be displayed from the perspective of the first object, and a real-world image of the first position can be displayed when the first object is in a first position. In this process, the 3D virtual image and the displayed image are combined to facilitate the monitoring personnel to intuitively understand the situation of the first path and to intuitively and accurately observe the behavior of the first object in the first position.

[0104] Figure 2 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 2 As shown, the first object is the handler for the alarm event. Before determining the first path from the first object to the first location, the method further includes:

[0105] In step S201, when an alarm event is detected, a 3D virtual image of the environment in which the alarm event occurs is displayed, and a marker of the alarm event is displayed in the 3D virtual image;

[0106] In step S202, upon receiving the first command, a 3D virtual image of the first location is displayed;

[0107] In step S203, upon receiving the second command, the handler and the second position of the handler are determined;

[0108] The determination of the first path from the first object to the first position includes:

[0109] In step S204, a first path from the second location to the first location is determined.

[0110] In one embodiment, the park monitoring method can be applied to scenarios involving alarm events, including but not limited to equipment malfunctions, objects being placed in restricted areas, not wearing masks, littering, abnormal workstations, the presence of irregular personnel in the area, and vehicles exceeding the permitted time limit in the area. These alarm events need to be handled, and the handler for handling alarm events can be a person or an object, such as a robot.

[0111] There are multiple ways to detect alarm events. For example, if the alarm event is a smoke alarm, it can be triggered by a smoke detector. If the alarm event is an object encroaching on a restricted area, an image acquisition device can be used to capture an image of the restricted area, and then the captured image can be identified to determine whether an object exists in the restricted area.

[0112] When an alarm event is detected, a 3D virtual image of the environment where the alarm event occurs can be displayed, along with a marker for the alarm event within the 3D virtual image. It should be noted that the interface displaying the 3D virtual image can show not only the primary location but also other content, such as a 2D image indicating the presence of an alarm event, with the primary location highlighted (e.g., emphasized) to allow monitoring personnel to take appropriate action based on the information provided.

[0113] In one embodiment, the monitoring personnel can input a first command by clicking on a prompt message or clicking on a displayed first location. After receiving the first command, a 3D virtual image of the first location can be displayed. The 3D virtual image of a first range near the first location can be displayed, and the first range can be scaled as needed, so that the monitoring personnel can have a general understanding of the situation near the first location.

[0114] The displayed 3D virtual image can contain only 3D virtual images, or it can include a real-world image of the first location on top of the 3D virtual image. For example, the real-world image can be integrated into the 3D virtual image of the first location. If the 3D virtual image of the first location contains multiple rooms, and the first location is specifically located in a target room in multiple directions, then the real-world image of the target room can be acquired by an image acquisition device in the target room, and then the real-world image can be integrated into the 3D virtual image to display the location of the target room. This allows monitoring personnel to intuitively and accurately see whether there are alarm events near the first location, as well as the severity of the alarm events.

[0115] Next, the monitoring personnel can enter a second command, such as by clicking a virtual button on the interface, such as the match handler button, to determine the handler who will handle the alarm event and the handler's second location. The method of determining the handler can be set as needed. For example, the method can be to determine the handler closest to the second location from among multiple handlers, or to determine an idle handler from among multiple handlers, and then determine the path from the second location to the first location as the first path.

[0116] Figure 3 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 3 As shown, the 3D virtual image displaying the first location includes:

[0117] In step S301, a second path is determined from the third position of the current monitoring viewpoint to the first position;

[0118] In step S302, a 3D virtual image moving along the second path is generated based on the current monitoring perspective.

[0119] In one embodiment, a second path can be determined from a third position to a first position from the current monitoring perspective. Then, a 3D virtual image moving along the second path can be generated based on the current monitoring perspective. Multiple frames of 3D virtual images can be generated continuously. These multiple frames of 3D virtual images can constitute a 3D video. The 3D video simulates the 3D virtual environment observed from the current monitoring perspective during the movement from the third position along the second path until the 3D virtual image of the environment around the first position is viewed, which helps monitoring personnel understand the situation of reaching the first position from the second path.

[0120] The current monitoring viewpoint can be the viewpoint displayed on the current interface. This viewpoint can be the viewpoint of a specific image acquisition device, or it can be a virtual viewpoint, such as an aerial viewpoint. The second path can be a virtual path and does not need to be provided to monitoring personnel or operators.

[0121] Figure 4 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 4 As shown, the 3D virtual image displaying the first location includes:

[0122] In step S401, the building where the alarm event is located is cut, split, and enlarged in the 3D virtual image of the environment where the alarm event is located to display the 3D virtual image of the first location.

[0123] In one embodiment, the building where the alarm event is located can be cut, split, and magnified in the 3D virtual image of the environment where the alarm event is located. For example, if the alarm event is located on the third floor of Building B in a building complex, Building B can be cut out from the building complex first, and then the cut-out Building B can be split. The splitting method can be set as needed. For example, the third floor of Building B can be highlighted, and then the building structure above the third floor can be peeled off to display the 3D virtual image of the first location, so that the monitoring personnel can intuitively see the situation of the first location of the alarm event.

[0124] Figure 5 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 5 As shown, the method further includes:

[0125] In step S501, as the first position changes over time, the trajectory of the change in the first position is recorded;

[0126] In step S502, a third position is determined based on the motion trajectory, wherein the third position is a position on the motion trajectory, or a position after the first position has changed;

[0127] The determination of the first path from the first object to the first position includes:

[0128] In step S503, a first path from the second position to the changed first position is determined.

[0129] In one embodiment, under certain circumstances, the first location of an alarm event may change over time. For example, if the alarm event is that there are unauthorized personnel moving around the park, the first location of such personnel will change.

[0130] In this scenario, the movement trajectory of the first position change can be recorded, and then a third position can be determined, wherein the third position is either the position on the movement trajectory or the position after the first position change. The path from the second position to the first position can then be used as the first path, which can be provided to the operator for navigation so that the operator can accurately reach the first position.

[0131] In the case where the third position is the changed first position, for example, the movement speed can be calculated based on the movement trajectory, and then the first position after a period of time can be predicted based on the speed and movement direction. The determined first path can be the first path from the second position to the changed first position, so that the handler can be accurately guided to the position where the irregular person will arrive based on the first path.

[0132] Figure 6 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 6 As shown, the method further includes:

[0133] In step S601, the target viewpoint is determined based on the motion trajectory of the first position;

[0134] In step S602, a 3D virtual image moving along the trajectory is generated based on the target viewpoint.

[0135] In one embodiment, the target viewpoint can be determined based on the motion trajectory of the first position. For example, the orientation of the target viewpoint is the direction of motion of the motion trajectory, and the height of the target viewpoint is the height of human eyes. Then, a 3D virtual image moving along the motion trajectory can be generated based on the target viewpoint. Multiple frames of 3D virtual images can be generated continuously. Multiple frames of 3D virtual images can constitute a 3D video. The 3D video simulates the 3D virtual environment observed from the target viewpoint during the motion along the motion trajectory, which is convenient for understanding the situation during irregular human movement.

[0136] Optionally, the real-world image displaying the first location includes:

[0137] The real-world image of the first location is fused and displayed in the 3D virtual image of the first location.

[0138] In one embodiment, when the first object is located at the first position, a real-world image can be fused and displayed in the 3D virtual image of the first position. For example, if the 3D virtual image of the first position contains an area with multiple rooms, and the first position is specifically located in a target room in multiple directions, then a real-world image of the target room can be acquired by an image acquisition device in the target room, and then the real-world image can be fused into the 3D virtual image to display the position of the target room. Other rooms outside the target room are still displayed as 3D virtual images, so that monitoring personnel can accurately determine the spatial position of the target room by combining the 3D virtual image, and intuitively and accurately observe the behavior of the first object at the first position based on the real-world image.

[0139] Figures 7A to 7M This is a schematic diagram illustrating an application scenario according to an embodiment of the present disclosure.

[0140] like Figure 7A As shown, the interface can display a 3D virtual image of the park. When an alarm event is detected, a 2D virtual button can be displayed on the interface, such as the "View Alarm Event" button on the left side of the interface. The type of alarm event can also be displayed as restricted area occupancy. Furthermore, an alarm event marker, such as a triangle marker, can be displayed in the 3D virtual image.

[0141] Users Figure 7A After clicking the "View Alarm Events" button, you can follow... Figures 7B to 7E First, determine the building where the alarm event is located. Figure 7B Then the building was divided and disassembled layer by layer. Figure 7C This allows us to determine the floor where the alarm occurred and then zoom in on it. Figure 7D and Figure 7E ), thereby displaying a 3D virtual image near the first location, for example Figure 7D and Figure 7E The location of the triangle marker is the first position, which can display 3D virtual images of the area around the triangle marker.

[0142] It should be noted that, in Figures 7B to 7E In the process, after the user clicks the "View Alarm Events" button, a real-time image of the first location can be displayed on the right side of the interface, so that the user can see whether there is an alarm event at the first location and the severity of the alarm event.

[0143] Furthermore, such as Figure 7F As shown, a real-world image of the first location can be displayed so that users can accurately understand the actual situation at the first location.

[0144] Next, the handler (i.e., the first object) responsible for handling the alarm event can be identified. A 3D virtual image of the handler moving along the first path can be generated based on the handler's perspective. Multiple frames of 3D virtual images can be continuously generated based on the handler's movement along the first path. These consecutive frames can form a 3D video, which simulates the 3D virtual environment observed from the handler's perspective as they move along the first path. The generated 3D video can then be sent to the handler to provide navigation.

[0145] Furthermore, such as Figure 7G As shown, the 3D virtual image in the interface can be scaled down first; then the interface can be moved near the operator to display the 3D virtual environment around the operator, for example... Figure 7H As shown; it is also possible to determine the first path from the location of the handler to the first location, for example Figure 7H A dashed line with an arrow in the middle; when the handler begins to move, the handler's position can be displayed in the 3D virtual environment, for example... Figure 7I and Figure 7J As shown; it is also possible to generate 3D virtual images based on the operator's perspective during the operator's movement, for example... Figure 7K As shown; finally, when the operator is in the first position, the real-world image can be fused and displayed in the 3D virtual image of the first position, for example... Figure 7L and Figure 7MAs shown, this allows users to observe the specific process by which the handler resolves the alarm event, such as the process of moving the box from the doorway as shown in the image.

[0146] Figure 8 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 8 As shown, the first object is a vehicle. Before determining the first path from the first object to the first location, the method further includes:

[0147] In step S801, a target parking space and a first location of the target parking space are determined in the parking lot;

[0148] The determination of the first path from the first object to the first position includes:

[0149] In step S802, a first path from the location of the vehicle to the first location is determined.

[0150] In one embodiment, when the first object is a vehicle, the method can be applied to scenarios where the vehicle enters a parking lot and parks. A target parking space and its first location can be determined within the parking lot. The target parking space can be an available parking space, or more specifically, the available parking space closest to the vehicle. The path from the vehicle's location to this first location can then be determined as a first path. Accordingly, the first path can be sent to the vehicle's driver to provide navigation.

[0151] Figure 9 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 9 As shown, the method further includes:

[0152] In step S901, the second object entering and exiting the vehicle, and the trajectory of the second object, are recorded and / or displayed.

[0153] In one embodiment, the movement of a second object entering or exiting the vehicle can be recorded and displayed. The second object can include a person or an object. The movement trajectory of the second object can also be recorded and displayed so that monitoring personnel can accurately understand who the second object entering or exiting the vehicle is, the destination of the second object, and other information.

[0154] Figures 10A to 10G This is a schematic diagram illustrating another application scenario according to an embodiment of the present disclosure.

[0155] like Figure 10AAs shown, the interface can display a 3D virtual image of the parking lot entrance. When a vehicle is detected that needs to enter, 2D virtual buttons can be displayed on the interface, such as the "Vehicle" and "Personnel" buttons on the left side of the interface. The interface can also display the image of the vehicle and its specific information, such as license plate and vehicle model.

[0156] Furthermore, such as Figure 10B As shown, the first location of the target parking space and the current location of the vehicle can be determined in the parking lot, and then the first path from the vehicle's current location to the first location can be determined (e.g., Figure 10B (As shown by the dashed line in the middle), where the first path can be sent to the vehicle to provide navigation for the vehicle.

[0157] It can also generate 3D virtual images of the vehicle moving along a first path from the vehicle's perspective. Furthermore, based on the vehicle's movement along the first path, multiple frames of 3D virtual images can be continuously generated. These consecutive frames can form a 3D video, which simulates the 3D virtual environment observed from the vehicle's perspective as it moves along the first path. The generated 3D video can then be sent to the vehicle to provide navigation.

[0158] During vehicle movement, a 3D virtual image of the vehicle moving along a first path can be generated from a third-person perspective. Furthermore, based on the vehicle's movement along the first path, multiple frames of 3D virtual images can be continuously generated, for example... Figure 10C and Figure 10D A series of consecutive 3D virtual images can form a 3D video. A 3D video is a 3D virtual environment observed from a third-person perspective as a vehicle moves along a first path, so that users can observe the vehicle's movement.

[0159] Then, when the vehicle is in the first position, the real-world image is fused and displayed in the 3D virtual image of the first position, for example... Figure 10E It displays images based on 3D virtual images so that users can see how their vehicles are parked in the parking spaces.

[0160] Next, the second object entering and exiting the vehicle, as well as the trajectory of the second object's movement, can be recorded and / or displayed, for example... Figure 10F and Figure 10G As shown, the second object is a person, whose specific information can be identified, such as department, name, gender, etc.

[0161] Figure 11 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 11 As shown, the method further includes:

[0162] In step S1101, a 3D virtual image of the first object moving along the first path is sent to the first object.

[0163] In one embodiment, the generated 3D virtual image of the first object moving along the first path can simulate the image seen from the perspective of the first object when it moves along the first path. By sending the 3D virtual image to the first object, navigation can be provided to the first object so that it can move accurately along the first path.

[0164] Figure 12 This is a schematic flowchart illustrating another park monitoring method according to an embodiment of this disclosure. Figure 12 As shown, the method further includes:

[0165] In step S1201, a 3D virtual image of the first object moving along the first path is recorded, and / or a real-world image of the first position is recorded when the first object is located at the first position.

[0166] In one embodiment, a 3D virtual image of the first object moving along a first path can be recorded, and a real-world image of the first position when the first object is in a first position can also be recorded. The recorded 3D virtual image and the displayed image can be time-related for subsequent querying.

[0167] The recorded content can have multiple dimensions, and each dimension can also include multiple sub-dimensions. When users view the recorded content, they can operate from the desired dimension or sub-dimension to view the recorded content from that dimension level.

[0168] For example, the recorded content can include three main dimensions: equipment assets, maintenance personnel, and work order status. Each main dimension can include multiple sub-dimensions, such as:

[0169] Equipment assets include three sub-dimensions: a list of maintenance work orders, a table of equipment operating status, and a view of equipment distribution.

[0170] The operations and maintenance personnel include three sub-dimensions: viewing the operations and maintenance work order list, viewing the operations and maintenance personnel list, and viewing the distribution of operations and maintenance personnel.

[0171] The work order status includes three sub-dimensions: operation and maintenance work order change curve, operation and maintenance work order list view, and operation and maintenance personnel distribution view.

[0172] Since the recorded content is time-related, a timeline can be displayed for easy querying. Clicking on a specific time point on the timeline will show the type of record if a record exists at that time, such as restricted area occupancy alarm events, equipment malfunction alarm events, or vehicles entering the parking lot. Furthermore, the aforementioned dimensions can also be displayed.

[0173] For example, if you select a device malfunction alarm event at that time and choose to view it from the sub-dimensional device operation status information table, you can display a 3D virtual image and / or real-world image of the device's location at that time. In the 3D virtual image, you can highlight the room where the device is located, for example, by using red or highlighting. You can also display the device's specific fault parameters, such as excessive temperature or damaged casing.

[0174] For example, if you select an equipment malfunction alarm event at that time and choose to view it from the sub-dimensional work order list, you can display a 3D virtual image from the perspective of the person handling the alarm at that time, as well as a 3D virtual image of the person's movement trajectory and a display image of the person handling the alarm event.

[0175] In addition to displaying the content in the above embodiments, the interface can also record and display other content as needed, such as recording the electricity consumption of each building, the number of users accessing the network, etc., and assigning different colors to the corresponding buildings according to the different recorded content so that users can view them.

[0176] In one embodiment, the generation of the aforementioned 3D virtual image can be achieved using 3D modeling technology.

[0177] Among them, it can combine CAD vector data and GIS information to realistically recreate the park's building data (buildings, park roads, conference rooms, parking lots, office areas, exhibition halls) and spatial relationships at a 1:1 scale.

[0178] It can also use the Unity3D engine for real-time rendering, providing multiple interaction methods (mouse and keyboard, touch, gesture control), giving users a free interactive experience in three-dimensional space, allowing users to view the park from a top-down perspective, or easily locate the area of ​​interest for micro-analysis.

[0179] It can also use IoT technology to acquire data from various smart devices, such as camera data (using artificial intelligence algorithms to analyze image data in real time, enabling one-step intelligent functions, such as electronic fences, passenger flow statistics, gesture recognition, and abnormal behavior marking), location data, meeting room usage status data, parking lot usage status data, etc. All data is uploaded to the cloud platform and uses the MQTT communication protocol to realize the subscription / publishing of data messages, achieving the goal of displaying 3D virtual images in real time.

[0180] Smart hardware devices (including but not limited to the aforementioned image acquisition devices) distributed throughout the park are connected to the network to collect data. The data is then transmitted and aggregated to the cloud platform server via message queue telemetry. After being analyzed and processed by artificial intelligence algorithms, specific business data is generated. Terminal devices can receive real-time message data pushes after subscribing.

[0181] The specific process can be as follows: the data acquisition module collects data from smart devices, uploads it to the cloud platform via MQTT, and generates corresponding business data after processing by artificial intelligence algorithms; the client's core control module connects to the cloud platform's Broker service via TCP and subscribes to relevant types of business messages. When the Broker has new data, it pushes it to subscribers in real time, allowing the client to receive the processed and generated business data in real time; after receiving the business data, the core control module analyzes and generates data for different business modules, such as changes in meeting room data, parking lot data, and abnormal alarm information, and then synchronously controls the refresh of the 2D UI and the linkage actions of the 3D entity model; the interaction control module captures user input, which is analyzed by the core control module and triggers corresponding display effects; the client can ultimately be presented on various media such as Windows large screens, Android mobile devices, and web pages, allowing users to view it anytime, anywhere.

[0182] After receiving the business data, the core control module analyzes and generates data for different business modules. Specifically, after receiving the subscribed business data, the Dispatcher organizes and classifies it into business module data such as meeting room, office area, parking lot, visitor, exhibition hall, and alarms, and then distributes it to different sub-processing units. The rendering unit displays the linkage between 2D UI and 3D entity model, and finally presents it on the terminal.

[0183] For example, in real-world scenarios, users can easily search for available parking spaces, reserve and lock them online, and navigate to the parking space with a single click, saving them the inconvenience of searching back and forth for a space. Alternatively, they can search for available meeting rooms, view the specific meeting room model to see the equipment configuration, reserve them online, and navigate to them with a single click. Or, if a smart device detects abnormal behavior in a restricted area, it can report it to the cloud platform, push it to the client, and ultimately generate a 3D alarm point displayed at the corresponding spatial location. Users can click to view details, enabling early detection and handling of abnormal behavior and improving the park's operational management capabilities and efficiency.

[0184] Corresponding to the embodiments of the above-described park monitoring methods, this disclosure also proposes embodiments of park monitoring devices.

[0185] Figure 13This is a schematic block diagram of a park monitoring device according to an embodiment of the present disclosure. The park monitoring device is applicable to a park monitoring system, which can be applied to scenarios such as buildings, parking lots, and office parks. The park monitoring system may include multiple image acquisition devices, such as cameras and video recorders. These image acquisition devices are positioned at different locations within the scene and can acquire images from various locations within the scene. The acquired images are then sent to the processor of the park monitoring system, which can process the received images, for example, converting them into 3D virtual images.

[0186] like Figure 13 As shown, the park monitoring device may include:

[0187] The path determination module 1301 is used to determine the first path from the first object to the first position;

[0188] The virtual image generation module 1302 is used to generate a 3D virtual image of the first object moving along the first path based on the perspective of the first object.

[0189] The real-view display module 1303 is used to display a real-view image of the first position when the first object is located at the first position.

[0190] Figure 14 This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 14 As shown, the first object is the handler for handling alarm events, and the device further includes:

[0191] The virtual image display module 1401 is used to display a 3D virtual image of the environment where the alarm event is located when an alarm event is detected, and to display a marker of the alarm event in the 3D virtual image; and to display a 3D virtual image of the first location when a first command is received;

[0192] The first position determination module 1402 is used to determine the handler and the second position of the handler when the second command is received;

[0193] The path determination module is used to determine a first path from the second location to the first location.

[0194] Optionally, the virtual image display module is used to determine a second path from the third position of the current monitoring viewpoint to the first position; and to generate a 3D virtual image that moves along the second path based on the current monitoring viewpoint.

[0195] Optionally, the virtual park monitoring module is used to cut, split, and enlarge the building where the alarm event is located in the 3D virtual image of the environment where the alarm event is located, so as to display the 3D virtual image of the first location.

[0196] Figure 15 This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 15 As shown, the device further includes:

[0197] The trajectory recording module 1501 is used to record the motion trajectory of the first position as it changes over time.

[0198] The position prediction module 1502 is used to determine a third position based on the motion trajectory, wherein the third position is a position on the motion trajectory, or a position after the first position has changed;

[0199] The path determination module is used to determine a first path from the second position to the changed first position.

[0200] Optionally, the virtual image generation module is further configured to determine the target viewpoint based on the motion trajectory of the first position; and to generate a 3D virtual image moving along the trajectory based on the target viewpoint.

[0201] Optionally, the real-scene display module is used to fuse and display the real-scene image of the first location into the 3D virtual image of the first location.

[0202] Figure 16 This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 16 As shown, the first object is a vehicle, and the device further includes:

[0203] The second location determination module 1601 is used to determine the target parking space and the first location of the target parking space in the parking lot;

[0204] The path determination module is used to determine a first path from the location of the vehicle to the first location.

[0205] Figure 17 This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 17 As shown, the device further includes:

[0206] The recording and display module 1701 is used to record and / or display a second object entering or exiting the vehicle, as well as the movement trajectory of the second object.

[0207] Figure 18This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 18 As shown, the device further includes:

[0208] Image sending module 1801 is used to send a 3D virtual image of the first object moving along the first path to the first object.

[0209] Figure 19 This is a schematic block diagram of another park monitoring device according to an embodiment of this disclosure. Figure 19 As shown, the device further includes:

[0210] The image recording module 1901 is used to record a 3D virtual image of the first object moving along the first path, and / or record a real-world image of the first position when the first object is located at the first position.

[0211] Embodiments of this disclosure also propose a campus monitoring system, comprising:

[0212] processor;

[0213] Memory used to store processor-executable instructions;

[0214] The processor is configured to implement the method described in any of the above embodiments.

[0215] Embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0216] In this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0217] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0218] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A park monitoring method characterized by, include: Determine a first path from a first object to a first location, where the first location is the location of the alarm event and the first object is the handler that handles the alarm event; A 3D virtual image of the first object moving along the first path is generated based on the perspective of the first object and sent to the first object. When the first object is located at the first position, display a real-world image of the first position; The method further includes: When the first location of the alarm event changes over time, record the movement trajectory of the change in the first location and determine the changed first location; The target viewing angle is determined based on the motion trajectory of the first position; wherein, the orientation of the target viewing angle is the direction of motion of the motion trajectory, and the height of the target viewing angle is at eye level; A 3D virtual image moving along the trajectory is generated based on the target viewpoint, and a 3D video is constructed based on multiple consecutive frames of 3D virtual images. The movement speed is calculated based on the movement trajectory, and the changed first position is predicted based on the movement speed and movement direction; the first path indicates the second position of the operator to the changed first position.

2. The method of claim 1, wherein, Before determining the first path from the first object to the first location, the method further includes: When an alarm event is detected, a 3D virtual image of the environment in which the alarm event occurs is displayed, and a marker for the alarm event is displayed in the 3D virtual image; Upon receiving the first command, display a 3D virtual image of the first location; Upon receiving the second command, the handler and its second location are determined; wherein determining the first path from the first object to the first location includes: Determine the first path from the second location to the first location.

3. The method of claim 2, wherein, The 3D virtual image displaying the first location includes: Determine a second path from the third position at the current monitoring viewpoint to the first position; A 3D virtual image moving along the second path is generated based on the current monitoring perspective.

4. The method of claim 2, wherein, The 3D virtual image displaying the first location includes: In the 3D virtual image of the environment where the alarm event is located, the building where the alarm event is located is cut, split, and enlarged to display the 3D virtual image of the first location.

5. The method of claim 1, wherein, The real-world image displaying the first location includes: The real-world image of the first location is fused and displayed in the 3D virtual image of the first location.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Record a 3D virtual image of the first object moving along the first path, and / or record a real-world image of the first position when the first object is located at the first position.

7. A park monitoring device characterized by comprising: include: The path determination module is used to determine the first path from the first object to the first location, wherein the first location is the location of the alarm event, and the first object is the handler that handles the alarm event; A virtual image generation module is used to generate a 3D virtual image of the first object moving along the first path based on the perspective of the first object, and send it to the first object; A real-view display module is used to display a real-view image of the first position when the first object is located at the first position; The device further includes: The virtual image display module is used to record the motion trajectory of the first position changing over time when the first position where the alarm event is located changes, and to determine the changed first position; to determine the target viewpoint based on the motion trajectory of the first position; to generate a 3D virtual image moving along the trajectory based on the target viewpoint, and to construct a 3D video based on multiple consecutive frames of 3D virtual images; The position prediction module is used to calculate the movement speed based on the movement trajectory, and predict the changed first position based on the movement speed and movement direction; the first path indicates the second position of the operator to the changed first position.

8. The apparatus of claim 7, wherein, The virtual image display module is also used to display a 3D virtual image of the environment where the alarm event is located when an alarm event is detected, and to display a marker of the alarm event in the 3D virtual image; And upon receiving the first command, display a 3D virtual image of the first location; The first position determination module is used to determine the handler and the second position of the handler when the second command is received; The path determination module is used to determine a first path from the second location to the first location.

9. The apparatus of any one of claims 7-8, wherein, The device further includes: An image sending module is used to send a 3D virtual image of the first object moving along the first path to the first object.

10. The apparatus according to any one of claims 7 to 8, characterized in that, The device further includes: The image recording module is used to record a 3D virtual image of the first object moving along the first path, and / or record a real-world image of the first position when the first object is located at the first position.

11. A park monitoring system, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 6.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 6.

Images