A method, device and medium for guiding intersection safety based on digital twin

By using digital twin technology to identify and predict the location and paths of vulnerable groups such as blind people at intersections, and adjust signal lights and voice broadcasts, it solves the problem of the existing system being difficult to effectively guide vulnerable groups in complex environments, and achieves a safe, portable, low-cost Cost-effective intersection safety guidance.

CN115880888BActive Publication Date: 2025-01-24FUDAN UNIVERSITY +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211503899.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-28
Publication Date
2025-01-24
Estimated Expiration
2042-11-28

Smart Images

  • Figure CN115880888B_ABST
    Figure CN115880888B_ABST
Patent Text Reader

Abstract

The present invention relates to a method, device and medium for safe guidance at intersections based on digital twins, and the method comprises the following steps: model construction: constructing a city intersection system and a vulnerable group model in 3D software; deployment and training of a neural network model: collecting image information of vulnerable groups at intersections from the perspective of a camera in the 3D software to obtain a training set for training; data acquisition and processing: the camera in the 3D software collects image data. The edge computing host processes the image. The result is transmitted to a virtual environment server and a physical traffic equipment control terminal; controlling traffic equipment: receiving signals, controlling traffic lights and audio equipment, and guiding the target through the intersection through voice broadcasting. The method can well solve the problem of safely guiding vulnerable groups at intersections. Compared with traditional methods, the method is low in cost, highly scalable, non-wearable and has a wide range of applications.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart transportation, and in particular to a digital twin-based intersection safety guidance method, device, and medium. Background Art

[0002] There are currently a large number of vulnerable groups represented by people with disabilities, but the technology products serving them are far from meeting their needs. In recent years, it has become a trend for technology to help people with disabilities integrate into society. Unfortunately, most of the assistive products produced under this trend are still immature and have not been widely used. There are still many gaps in this field that need to be filled. Taking the blind as an example, the following explains some common deficiencies of assistive technology products for vulnerable groups through the deficiencies of existing blind assistance products.

[0003] Taking the disadvantaged group of blind people as an example, most of the research on traditional blind assistance products is on smart blind sticks and smart glasses. The principle is to use a combination of multiple sensors such as lidar, ultrasonic ranging sensors, etc., and use the Internet and GPS positioning to help the visually impaired achieve obstacle avoidance, navigation and other functions. Most of these blind assistance products have a single function and the actual use effect is not good. The positioning of the navigation function is always biased when used in actual scenes, which is easy to mislead the visually impaired. The role of sensors is mostly to assist the visually impaired to avoid obstacles and ensure the safety of the visually impaired when walking. In recent years, many researchers have continuously added functions to guide sticks based on the Internet of Things era, such as adding acceleration sensors combined with specific algorithms to achieve anti-fall monitoring functions, adding ultrasonic ranging modules combined with algorithms to achieve deep pit detection, etc., but the practical effect is not ideal. With the rise of computer vision technology, blind-assistance products such as smart visual aids that use computer vision technology to assist the visually impaired have begun to appear in the field of blind guides. Most of these products use computer vision and intelligent voice technology to convert visual information into auditory information. The principle is mostly to capture the external environment through a camera system, detect and identify the captured images, and then feed back the information to the visually impaired. Such products can make up for the shortcomings of previous products. When used in combination with GPS, they can also achieve intelligent navigation. In recent years, some experts have tried to apply environmental perception technology in the field of autonomous driving to smart visual aids to further develop blind guide products. Later, researchers also added cameras to the original smart blind sticks, and after training, they identified blind paths and crosswalks, thereby achieving the effect of accurately guiding the visually impaired. However, there are not many companies researching such products at present, resulting in the immaturity of this technology, and such products have not been able to play their advantages well in many complex actual scenarios such as intersections. Although some researchers have made improvements to the products for specific scenarios, it is still difficult to meet the actual use needs. More importantly, most of these products are wearable, which puts a burden on the body of the visually impaired when using them. This also leads to extremely high requirements for portability and weight of these blind-assistance products, and blind-assistance products such as smart visual aids have limited computing power. On the other hand, these products are expensive, and most visually impaired people cannot afford to buy them, which limits the user group and makes it impossible to serve most visually impaired people well. Represented by blind-assistance products, most of the current assistive technology products serving vulnerable groups are not very portable and expensive, and the existing related auxiliary functions are difficult to guide users in the complex environment of urban intersections.

[0004] For the scene of intersections, the field of smart transportation has achieved initial results. In recent years, with the rapid development of technologies in many fields including digital twins, smart transportation has gradually emerged. The current mainstream development direction of smart transportation is to adjust the duration of traffic lights in real time according to the traffic volume, so as to effectively solve the problem of traffic congestion. In fact, the field of smart transportation can also provide solutions to solve the problem of safety guidance for vulnerable groups in intersection scenarios.

[0005] Traditional smart transportation mainly focuses on traffic flow at intersections. The principle is to use intersection radars, cameras and other sensing equipment, and integrate high-precision positioning technology, high-precision map technology, edge computing technology, etc. to obtain information about vehicles at intersections. On this basis, it generates a variety of accurate and real-time metadata such as vehicle trajectory, passing vehicle identity, and traffic light status. These data are analyzed through specific algorithms to ultimately adjust the duration of traffic lights, thereby automatically adjusting traffic lights according to traffic flow.

[0006] In summary, the existing intersection safety guidance system has the following shortcomings:

[0007] (1) Most assistive technology products that serve vulnerable groups are not portable and are expensive. Existing assistive functions are difficult to guide users in the complex environment of urban intersections.

[0008] (2) There is a lack of an intersection guidance system that combines digital twin technology to achieve safe guidance at intersections. Summary of the invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a digital twin-based intersection safety guidance method, device, and medium, which can provide safety guidance for a wide range of service objects while having strong scalability.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] One aspect of the present invention provides a method for guiding intersection safety based on digital twins, comprising the following steps:

[0012] Acquire image information of at least one image acquisition device at the target intersection, convert the image information into a preset format and input it into a pre-trained neural network model, acquire the location and predicted path of the target person and synchronize them to a preset virtual environment;

[0013] Sending action signals to the signal lights and / or voice broadcasting devices in the actual environment and the virtual environment according to the position of the target person, the predicted path and the preset rules to complete the safety guidance of the target person;

[0014] At preset time intervals, a new training set is obtained according to the virtual environment to update the neural network model.

[0015] As a preferred technical solution, the process of establishing the virtual environment includes the following steps:

[0016] The environmental information of the target intersection is obtained and imported into the modeling software, pedestrian and vehicle models are added and the logic between the vehicle and the traffic light is established, a blind model and a virtual camera matching the position and parameters of the image acquisition device in the actual environment are added to the modeling software to establish the virtual environment.

[0017] As a preferred technical solution, the environmental information is obtained via satellite and / or drone.

[0018] As a preferred technical solution, the process of obtaining the pre-trained neural network model includes the following steps:

[0019] Obtain a training sample set, wherein each sample in the training sample set includes a sample image and a corresponding image with a target person's position annotated;

[0020] The neural network model is trained based on the training sample set, and after the loss function value reaches a preset convergence condition, the pre-trained neural network model is obtained,

[0021] The process of obtaining the training sample set includes:

[0022] In the virtual environment, a plurality of sample images are acquired by using at least one virtual camera preset in the virtual environment;

[0023] According to the original image, annotating the area where the target person is located to obtain a picture with the location of the target person marked;

[0024] The sample images and the images with the target person's position marked are converted into a preset format to obtain the training sample set.

[0025] As a preferred technical solution, the acquisition of the target person's location and predicted path includes the following steps:

[0026] According to the acquired image information, the coordinates of the zebra crossing are determined, and the coordinate functions of each line of the zebra crossing are calculated, so as to divide multiple areas and establish a mapping relationship from the image to the actual environment;

[0027] Obtain the coordinates of the rectangular frame of the target person and calculate the center coordinates;

[0028] According to the center coordinates and the coordinate function and the position of the image source image acquisition device, use linear programming to determine the area and position of the target person, and obtain a predicted path according to the area and position of the target person;

[0029] The target person's position and predicted path are synchronized into the virtual environment.

[0030] As a preferred technical solution, the preset rules include:

[0031] Determine whether the target person is recognized in the image information. If the target person is not recognized, the traffic lights operate according to normal rules. If the target person is recognized and is about to enter the intersection, the traffic lights on the sidewalk in the direction of the target person in the actual environment and the virtual environment are turned green, and the traffic lights on the lanes in the other direction are turned red. The remaining traffic lights operate according to normal rules until the target person passes the sidewalk;

[0032] It is determined whether the target person is recognized in the image information. If the target person is not recognized, the voice broadcast device does not operate. If the target person is recognized, guidance suggestion information is issued according to the relative position of the target person and the sidewalk until the target person passes the sidewalk.

[0033] As a preferred technical solution, the pre-trained neural network model is deployed on an edge computing device.

[0034] As a preferred technical solution, the virtual environment is established using Unreal Engine 5 or Unity.

[0035] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the above-mentioned digital twin-based intersection safety guidance method.

[0036] Another aspect of the present invention provides a computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the above-mentioned digital twin-based intersection safety guidance method.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) For each specific intersection, a virtual model of the intersection is constructed by mapping through satellite or drone means, and pedestrians, vehicles and other elements are added to the virtual environment in combination with the actual situation. On this basis, target vulnerable groups are added to the pedestrian crowd, such as the blind, pregnant women, and people with disabilities. Due to the use of digital twin technology, the recognition model is trained in the virtual environment without obtaining the training set of the neural network from the actual environment, which reduces the difficulty of training the neural network and reduces the difficulty of implementing the present invention.

[0039] (2) By calling the camera images of the real intersection, the existence of the target vulnerable group is determined and the corresponding path is planned. The traffic lights are adjusted according to the processing results, and the audio equipment is controlled to broadcast the recommended direction of travel to the target vulnerable group, thereby achieving safe guidance in the scene of the intersection. It can guide a variety of target people including the blind, pregnant women, and disabled people, and has high practicality.

[0040] (3) There is no need to collect a large amount of data in the actual environment to train the neural network. At the same time, the existing method of using wearable guidance devices to guide target personnel is abandoned, providing a non-wearable, low-cost, highly scalable, and widely used intersection safety guidance method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the intersection safety guidance method based on digital twin in Example 1;

[0042] Figure 2 This is a diagram of the overall process framework of intersection safety guidance in Example 2;

[0043] Figure 3 This is a flowchart of the overall process of intersection safety guidance in Example 2;

[0044] Figure 4 This is an example of a blind person model constructed by 3D software in step S1 of Example 2;

[0045] Figure 5 The intersection model constructed by the 3D software in step S1 of embodiment 2;

[0046] Figure 6 The intersection model in the virtual city background constructed by the 3D software in step S1 of embodiment 2;

[0047] Figure 7 This is a flowchart of the software process of the image processing part in step S3 of Example 2;

[0048] Figure 8 It is the result of processing the image information obtained from the camera's viewing angle when a blind person appears at the intersection in step S3 of embodiment 2. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0050] Example 1

[0051] like Figure 1 The present embodiment provides a digital twin-based intersection safety guidance method, comprising the following steps:

[0052] Step S1, model construction:

[0053] Use satellite or drone mapping to construct the city’s intersection system and build a model for the blind.

[0054] Step S2, deployment and training of neural network model:

[0055] In the 3D software, the information of blind people at the intersection is collected from the camera's perspective to obtain the training set for training.

[0056] Step S3, data collection and processing:

[0057] The camera in the 3D software collects image data. The edge computing host processes the image. The results are transmitted to the virtual environment server and the physical model control terminal.

[0058] Step S4, controlling traffic equipment:

[0059] Receive signals, control traffic lights and audio equipment, and guide targets through intersections through voice broadcasts.

[0060] Wherein, step S1 specifically includes the following steps:

[0061] Step S11, selecting a mature 3D modeling software (such as Unreal Engine 5, Unity, etc.), and importing the urban intersection system that has been mapped by satellite or drone, etc., into the software, including the complete intersection and traffic lights and other elements;

[0062] Step S12, adding pedestrian and vehicle models to the virtual environment and setting the logic between the vehicles and traffic lights, so that the entire environment can achieve the same effect as in reality when it is running;

[0063] Step S13, constructing a blind person model with features such as wearing sunglasses and holding a cane in the virtual environment, adding a walking animation of the blind person to the virtual environment;

[0064] Wherein, step S2 includes:

[0065] Step S21, constructing multiple cameras based on the intersection model, adjusting the position parameters of the cameras, and observing the intersection from the perspective of the cameras;

[0066] Step S22, using a camera to acquire an image from the virtual environment;

[0067] Step S23, using data annotation software (such as labelImg, etc.) to annotate the image;

[0068] Step S24, using a deep neural network for training.

[0069] Step S25, deploy the trained neural network model on the edge computing host, and then place it at a real city intersection.

[0070] Wherein, step S3 includes:

[0071] Step S31, in reality, the image information acquired by the camera is retrieved and transmitted to the edge computing host;

[0072] Step S32, completing the recognition of the blind person in the image and obtaining the coordinate information of the blind person in the image. If the blind person is not recognized, the steps S31 and S32 are continuously cycled;

[0073] Step S33, calculating the area and position of the blind person;

[0074] Step S34, determining the current state of the blind person and giving corresponding control instructions;

[0075] Step S35, transmitting the control instruction to the virtual environment server and the physical intersection control terminal.

[0076] Wherein, step S4 includes:

[0077] Step S41, according to the received result, the virtual environment end and the physical model control end simultaneously control the traffic light to change accordingly;

[0078] Step S42, based on the received result, the virtual environment end and the physical model control end simultaneously control the audio device to broadcast the corresponding instructions to guide the target to pass the intersection safely.

[0079] After a preset time interval, the training set is re-acquired according to the virtual environment and the neural network model is updated.

[0080] Example 2

[0081] like Figures 2 to 7 As described above, this embodiment provides a safety guidance system for intersections. Taking the target vulnerable group as the blind, the 3D software using Unreal Engine 5, and the edge computing host using the Xinbu GNS-V40 host as an example, the implementation method is specifically described.

[0082] Step S1, model construction:

[0083] Construct the city intersection system and blind model in 3D software. Build the intersection entity model.

[0084] Step S1 specifically includes the following operations:

[0085] Step S11, importing into Unreal Engine 5 a city intersection system model that has been mapped by satellite or drone, etc., wherein the model includes various elements in the actual scene, wherein the necessary parts include a complete intersection consisting of motor vehicle lanes and sidewalks, traffic lights in four directions of east, west, south, and north, four street lights at the intersection, etc., and other non-essential elements include roadside vegetation, road signs, buildings, etc.;

[0086] Step S12, construct or directly import open source character models and vehicle models in Unreal Engine 5 software, change the skeleton and clothing of the character models, obtain different types of character models such as men, women, old and young, and then construct pedestrian animations to bind them, and then place a number of character models at appropriate locations in the constructed urban environment to act as pedestrians; construct vehicle driving animations to bind them to vehicle models, and then place a number of vehicle models at appropriate locations in the urban environment; finally, construct event logic to bind pedestrians, vehicles and traffic light status changes on the zebra crossing together, so that both pedestrians and vehicles have the effect of "stop at red light and go at green light";

[0087] Step S13, modifying the skeleton and dress of the constructed character model in Unreal Engine 5 software, so that the character model is in a state of wearing sunglasses, with the right hand bent in front of the body and holding a cane, and then constructing a walking animation and a cane swinging animation to bind it, so that when it runs, it has the effect of the body slightly leaning forward, holding the cane in front of the body to constantly explore the way, and moving forward slowly. Finally, a number of blind models are placed at appropriate locations in the constructed urban environment, and event logic is constructed to bind the blind person's moving state and moving direction to the state change of the traffic light and the virtual environment respectively, so that the state and time of the traffic light can be affected by the forward state of the blind person and the virtual environment can control the forward direction of the blind person.

[0088] Step S2, deployment and training of neural network model:

[0089] In the 3D software, the information of blind people at the intersection is collected from the camera's perspective to obtain the training set for training.

[0090] Step S2 specifically includes the following operations:

[0091] Step S21, constructing four cameras in Unreal Engine 5 software, adjusting the positions and parameters of the cameras so that the cameras are close to the positions and angles of cameras at intersections in real life.

[0092] Step S22, using the built-in screenshot program in Unreal Engine 5 software, collecting a large number of images from the perspective of the camera, and changing different weather (sunny, rainy) and different lighting (day, night) to collect a large number of images.

[0093] Step S23, labelImg software is used to label the image, the area of ​​the blind person in the image is selected with a tightly fitting rectangular frame, and then the coordinates of the upper left corner and the lower right corner of the rectangle are obtained and labeled as blind.

[0094] Step S24: Install PyTorch on the edge computing host, install the Python library functions required for running the Yolo v5 algorithm, and install Intel Extension for Pytorch. Use TorchVision, a data augmentation library officially provided by PyTorch, to perform data augmentation. Each time, the sample size of the data set is amplified by 10 times, and the image color, size, shape, space, and pixel angles are transformed, and then deep learning training is performed.

[0095] Step S25, deploy the trained model on the edge computing host, place the host at a real intersection, and connect the control terminals of various traffic devices.

[0096] Step S3, data collection and processing:

[0097] The camera at the real intersection collects image data. The edge computing host processes the image. The result is transmitted to the virtual environment server and the real intersection control terminal.

[0098] Step S3 specifically includes the following operations:

[0099] Step S31, retrieve the image data from the camera and transmit it to the edge computing host.

[0100] Step S32, calibrate the image, obtain the coordinates of the four corners of the zebra crossing, calculate the coordinate function of each line of the zebra crossing, thereby dividing each area and establishing a mapping relationship from the image to the real space.

[0101] Step S33, use the Yolo v5 algorithm to infer the image, obtain the coordinates of the upper left corner and the lower right corner of the rectangular frame of the blind person identified, and calculate the center coordinates

[0102] Step S34, according to the coordinates of the blind person, the coordinate function of the zebra crossing and the position of the image source camera, linear programming is used to determine the area and position of the blind person. According to the position of the blind person, the path of the blind person is planned in combination with the corresponding algorithm, and the corresponding processing result is obtained.

[0103] Step S35: sending corresponding control instructions to the virtual environment end and the physical intersection control end through the MQTT protocol according to the processing result.

[0104] Step S4, controlling traffic equipment:

[0105] Receive signals, control traffic lights and audio equipment, and guide targets through intersections through voice broadcasts.

[0106] Step S4 specifically includes the following operations:

[0107] Step S41: Based on the blind person identification information in the received result, if the blind person is not identified or the blind person is identified but has not moved to the area about to enter the intersection, the traffic light operates according to the normal rules; if the blind person is identified and is about to enter the intersection, the virtual environment end and the physical model control end simultaneously control the traffic light to change accordingly according to the direction of travel information in the result, the pedestrian traffic light in the direction of the blind person turns green, and the traffic light on the lane in the other direction turns red, and the remaining traffic lights continue to operate according to the original rules until the result information that the blind person has left is received;

[0108] Step S42, based on the received result, if the blind person is not identified, the voice will not be broadcast. If the blind person is identified, the virtual environment end and the physical model control end simultaneously control the audio device to broadcast a pre-recorded voice prompt to pedestrians that there is a blind person at the intersection, and pedestrians are requested to pay attention to avoid it, and instruct the blind person to follow the direction instructions, and then broadcast the direction instructions of "please move left" or "please move right" to guide the blind person to always walk in the center of the zebra crossing until they pass the intersection. The audio device will broadcast the voice of "the visually impaired person has passed, and the traffic light has returned to normal."

[0109] This embodiment combines the actual situation that most intersections are currently equipped with cameras and voice broadcasting equipment, and comprehensively uses deep learning, computer vision and other technologies to build a set of intersection safety guidance system based on digital twin technology. This system will extract image information from the cameras installed at the intersection, detect and identify the target vulnerable groups, and work with the traffic system to help the target cross the road. This system does not require the target vulnerable groups to carry any auxiliary equipment, and can be combined with existing public resources, does not require the vulnerable groups to pay any fees, and in theory can serve almost all target vulnerable groups. The development of this system can, on the one hand, allow vulnerable groups such as the disabled to enjoy the benefits brought by modern technology and reduce their burdens, and on the other hand, it can also make up for the gap in the actual scene of the intersection of related auxiliary products for vulnerable groups, and better promote the development of related products. This system is actually fully scalable. At the same time, the system can also be integrated into the smart city digital twin system to promote the development of smart cities, and it is also of great significance to promote the development of holographic intersections and smart transportation.

[0110] Example 3

[0111] This embodiment provides an electronic device, including one or more processors and a memory, wherein one or more programs are stored in the memory, and the one or more programs include instructions for executing the intersection safety guidance method based on digital twins as described in Example 1.

[0112] Example 4

[0113] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the intersection safety guidance method based on digital twins as described in Example 1.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A digital twin-based intersection safety guidance method, characterized in that: The steps include: Acquire image information of at least one image acquisition device at the target intersection, convert the image information into a preset format and input it into a pre-trained neural network model, acquire the location and predicted path of the target person and synchronize them to a preset virtual environment; Sending action signals to the signal lights and / or voice broadcasting devices in the actual environment and the virtual environment according to the position of the target person, the predicted path and the preset rules to complete the safety guidance of the target person; At preset intervals, a new training set is obtained according to the virtual environment to update the neural network model, The method of obtaining the target person's location and predicted path includes the following steps: According to the acquired image information, the coordinates of the zebra crossing are determined, and the coordinate functions of each line of the zebra crossing are calculated, so as to divide multiple areas and establish a mapping relationship from the image to the actual environment; Obtain the coordinates of the rectangular frame of the target person and calculate the center coordinates; According to the center coordinates and the coordinate function and the position of the image source image acquisition device, use linear programming to determine the area and position of the target person, and obtain a predicted path according to the area and position of the target person; Synchronize the target person's position and predicted path into the virtual environment, The preset rules include: Determine whether the target person is recognized in the image information. If the target person is not recognized, the traffic lights operate according to normal rules. If the target person is recognized and is about to enter the intersection, the traffic lights on the sidewalk in the direction of the target person in the actual environment and the virtual environment are turned green, and the traffic lights on the lanes in the other direction are turned red. The remaining traffic lights operate according to normal rules until the target person passes the sidewalk; It is determined whether the target person is recognized in the image information. If the target person is not recognized, the voice broadcast device does not operate. If the target person is recognized, guidance suggestion information is issued according to the relative position of the target person and the sidewalk until the target person passes the sidewalk.

2. According to claim 1, a digital twin-based intersection safety guidance method is characterized in that: The process of establishing the virtual environment includes the following steps: The environmental information of the target intersection is obtained and imported into the modeling software, pedestrian and vehicle models are added and the logic between the vehicle and the traffic light is established, a blind model and a virtual camera matching the position and parameters of the image acquisition device in the actual environment are added to the modeling software to establish the virtual environment.

3. The intersection safety guidance method based on digital twin according to claim 2 is characterized in that: The environmental information is obtained via satellite and / or drone.

4. The intersection safety guidance method based on digital twin according to claim 1 is characterized in that: The process of obtaining the pre-trained neural network model includes the following steps: Obtain a training sample set, wherein each sample in the training sample set includes a sample image and a corresponding image with a target person's position annotated; The neural network model is trained based on the training sample set, and after the loss function value reaches a preset convergence condition, the pre-trained neural network model is obtained, The process of obtaining the training sample set includes: In the virtual environment, a plurality of sample images are acquired by using at least one virtual camera preset in the virtual environment; According to the sample image, an image with the target person's location marked is obtained by marking the area where the target person is located; The sample images and the images with the target person's position marked are converted into a preset format to obtain the training sample set.

5. The intersection safety guidance method based on digital twin according to claim 1 is characterized in that: The pre-trained neural network model is deployed on the edge computing device.

6. The intersection safety guidance method based on digital twin according to claim 1 is characterized in that: The virtual environment is created using Unreal Engine 5 or Unity.

7. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the digital twin-based intersection safety guidance method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the digital twin-based intersection safety guidance method as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Intersection pass early warning method based on computer vision

    CN106373430A

  • Parallel intelligent vehicle control system based on ACP (artificial systems, computational experiments and parallel execution)

    CN107272683A