Simulation system, method, device and medium based on semi-physical simulation

Abstract

This application provides a simulation system, method, device, and medium based on hardware-in-the-loop (HIL) simulation. The system includes a control system, a transport device, and obstacles. The transport device sends its location information to the control system. The control system receives obstacle layout information from a virtual scene, performs layout calculations based on the obstacle layout information and the device location information, obtains layout control commands, and sends the layout control commands to the transport device. The transport device also transports the obstacles according to the layout control commands, so that the site after transport presents an obstacle layout corresponding to the obstacle layout information. This can at least solve the technical problems of high site requirements and high scene deployment costs associated with using HIL simulation in related technologies.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a system, method, apparatus and computer-readable medium for virtual content. Background Technology

[0002] Virtual Reality (VR) is a simulation technology that uses computer systems to generate realistic simulated environments, immersing users in them. Some VR experiences require wearing a virtual reality head-mounted display, also called a VR headset or simply a VR headset. VR headsets use a head-mounted display to block out a person's vision and hearing, creating the feeling of being in a virtual environment.

[0003] Currently, there are two main types of immersive scenarios that require wearing virtual reality headsets:

[0004] The first approach is to guide users to experience various virtual scenarios in open, flat spaces to create an immersive experience. The second approach is to use semi-physical simulations to create an immersive experience. Semi-physical simulations refer to physical objects, such as half-height walls, that correspond to the architectural layout in the virtual scenario. Interactive devices, such as buttons and switches, can also be configured based on these physical objects to provide users with a stronger sense of immersion.

[0005] However, the inventors have discovered at least the following technical problems in the related technology:

[0006] In the first type of immersive scenario, since the walls and other obstacles in the virtual scene do not have corresponding representations in the real space, the experiencer can freely or unintentionally "penetrate" the non-existent "walls," resulting in a decrease in immersion. In the second type of immersive scenario, since it is necessary to arrange physical objects in the actual scene that correspond to the architectural layout in the virtual scene, the requirements for the site are very high. Moreover, once the architectural layout in the virtual scene changes, the original layout is no longer applicable. That is, the original layout cannot adapt to the virtual scene after the architectural layout has changed, resulting in high scene deployment costs. Summary of the Invention

[0007] One objective of this application is to provide a simulation system, method, device, and medium based on hardware-in-the-loop simulation, at least to solve the technical problems of high site requirements and high scene deployment costs associated with using hardware-in-the-loop simulation in related technologies.

[0008] To achieve the above objectives, some embodiments of this application provide a simulation system based on hardware-in-the-loop simulation. The system includes a control system, a transport device, and obstacles. The transport device is used to send its device position information to the control system. The control system is used to receive obstacle layout information from a virtual scene, perform layout calculations based on the obstacle layout information and the device position information, obtain layout control instructions, and send the layout control instructions to the transport device. The transport device is also used to transport the obstacles according to the layout control instructions, so that the site after transport presents an obstacle layout corresponding to the obstacle layout information.

[0009] Some embodiments of this application also provide a simulation method based on hardware-in-the-loop simulation, applied to the system described above. The method includes: acquiring the equipment position information of the transport equipment; acquiring obstacle layout information of a virtual scene, and performing layout calculations based on the obstacle layout information and the equipment position information to obtain layout control instructions; controlling the transport equipment to transport the obstacles according to the layout control instructions, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information.

[0010] Some embodiments of this application also provide a simulation device based on hardware-in-the-loop simulation, the device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the method described above.

[0011] Some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method.

[0012] Compared to existing technologies, the solution provided in this application embodiment obtains the device location information uploaded by the transport equipment and the obstacle layout information of the virtual scene from the control system. Then, layout calculations are performed based on the obstacle layout information and the device location information to obtain layout control instructions. These instructions are then sent to the transport equipment, enabling it to transport the obstacles according to the instructions. Consequently, the site after transport can present an obstacle layout corresponding to the obstacle layout information. On one hand, since the positions of the obstacles in this application embodiment are variable and flexibly set, the requirements for the site are reduced. On the other hand, since obstacles can be automatically transported based on the obstacle layout information of the virtual scene to form the required obstacle layout, the same site can meet the obstacle layout requirements of multiple virtual scenes. In other words, even if the building layout in the virtual scene changes, the obstacle layout corresponding to the actual site will automatically adjust to adapt to the change, thereby reducing scene deployment costs. Attached Figure Description

[0013] Figure 1 An exemplary schematic diagram of the obstacle layout corresponding to the obstacle layout information is presented on the site after the handling equipment completes the handling in a simulation system based on semi-physical simulation provided in this application embodiment;

[0014] Figure 2 An exemplary schematic diagram illustrating the handling of a transport device according to a transport path in a simulation system based on a semi-physical simulation provided in this application embodiment;

[0015] Figure 3 An exemplary schematic diagram of an AGV in a hardware-in-the-loop simulation system provided for embodiments of this application;

[0016] Figure 4 An exemplary side view of an AGV handling wall panel in a simulation system based on semi-physical simulation, provided for an embodiment of this application;

[0017] Figure 5 An exemplary top view of an AGV transport wall panel in a simulation system based on semi-physical simulation, provided for an embodiment of this application;

[0018] Figure 6 An exemplary flowchart of a simulation method based on hardware-in-the-loop simulation provided in this application embodiment;

[0019] Figure 7 This is an exemplary structural diagram of a simulation device based on hardware-in-the-loop simulation provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a simulation system based on hardware-in-the-loop simulation, the system including a control system, a handling device, and obstacles;

[0022] The conveying device is used to send its device location information to the control system;

[0023] The control system is used to receive obstacle layout information of the virtual scene, perform layout calculation based on the obstacle layout information and the device position information, obtain layout control instructions, and send the layout control instructions to the handling device.

[0024] The transport equipment is also used to transport the obstacle according to the layout control command, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information.

[0025] It is possible Figure 1 and Figure 2 As shown, Figure 1 An exemplary schematic diagram showing the obstacle layout corresponding to the obstacle layout information of the site after the handling equipment has completed its handling; Figure 2 An exemplary schematic diagram illustrating the movement of equipment according to a moving path.

[0026] Specifically, after receiving obstacle layout information from the virtual scene, the control system can perform layout calculations based on the obstacle layout information and the equipment position information of the handling equipment. For example, if the venue in the virtual scene is rectangular, the obstacle layout information in the virtual scene can be the obstacle layout information for placing obstacles at the intersection of the diagonals of the venue in the virtual scene. Corresponding to the actual venue, if the actual venue is also rectangular, then an obstacle needs to be placed at the intersection of the diagonals of the rectangle. The control system will obtain the position information of where the obstacle should be placed, that is, the intersection of the diagonals of the rectangle mentioned above. If the actual venue is circular or other irregular shape, then it will be automatically divided into rectangles based on the circle or irregular shape, and the intersection of the diagonals of the rectangle will be obtained, and an obstacle will be placed at the intersection of the diagonals of the rectangle.

[0027] Furthermore, the control system acquires the location information of the transport equipment. It can be understood that there can be one or more transport equipment. If there is only one transport equipment, the control system acquires the location information of that single transport equipment, because it needs to plan the transport path based on the location information of that equipment and the obstacle layout information. If there are multiple transport equipment, the control system acquires the location information of all of them, determines the location of the most convenient transport equipment from among the multiple locations based on the obstacle layout information, and plans the transport path based on the location information of the most convenient transport equipment and the obstacle layout information.

[0028] In some embodiments of this application, the transport device can correspond one-to-one with the obstacle; the transport device is specifically used to transport the obstacle corresponding to the transport device according to the layout control command, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information.

[0029] For example, suppose there are 3 transport devices and 3 obstacles, with a one-to-one correspondence between the transport devices and the obstacles. These are transport device 1, transport device 2, and transport device 3; obstacles 1, obstacles 2, and obstacles 3. The location information for obstacle 1 is IP 1, the location information for obstacle 2 is IP 2, and the location information for obstacle 3 is IP 3. Then, the obstacle location information can be pre-stored in the transport devices: the pre-stored obstacle location information in transport device 1 can be (obstacle 1, IP 1), the pre-stored obstacle location information in transport device 2 can be (obstacle 2, IP 2), and the pre-stored obstacle location information in transport device 3 can be (obstacle 3, IP 3). Therefore, after receiving the layout control command, the transport device can move the obstacle corresponding to that device, so that the site after the movement presents an obstacle layout corresponding to the obstacle layout information.

[0030] In some examples, the transport equipment and the obstacle can be rigidly connected one-to-one based on structural components. In this way, since the transport equipment and the obstacle are integrated into a single design, the transport equipment does not need to pre-store the obstacle's location information. It only needs to transport the obstacle rigidly connected to the corresponding transport equipment after receiving the layout control command, so that the site after transport presents an obstacle layout corresponding to the obstacle layout information.

[0031] In some embodiments of this application, one transport device corresponds to multiple obstacles; the obstacles are used to send their obstacle position information to the control system; the control system is specifically used to receive obstacle layout information of a virtual scene, and perform layout calculations based on the obstacle layout information of the virtual scene, the device position information, and the obstacle position information to obtain layout control instructions, and send the layout control instructions to the transport device; wherein, the layout control instructions include instruction information of the obstacles to be controlled by the transport device.

[0032] For example, suppose there is one transport device and three obstacles, with one transport device corresponding to three obstacles, namely transport device 1, obstacle 1, obstacle 2, and obstacle 3. The location information of obstacle 1 is IP 1, the location information of obstacle 2 is IP 2, and the location information of obstacle 3 is IP 3. Then, obstacle 1, obstacle 2, and obstacle 3 can each send their respective location information to the control system. In this way, the control system can perform layout calculations based on the obstacle layout information of the virtual scene, the device location information, and the obstacle location information. For example, if transport device 1 takes the shortest time to transport obstacle 2, then an instruction message carrying a layout control command instructing transport device 1 to transport device 1 is sent to transport device 1, so that the obstacle to be controlled by transport device 1 is obstacle 2.

[0033] In some cases, a one-to-many connection can be established between the handling equipment and the obstacle based on a robotic arm. In this way, the handling equipment can grasp the obstacle and carry it by means of the robotic arm set on the handling equipment.

[0034] In some examples, the transport equipment may be, but is not limited to, an Automated Guided Vehicle (AGV), such as... Figure 3 As shown. In practical applications, because the AGV itself is equipped with batteries, motors, and other mechanisms, its own weight is relatively stable, thus ensuring stability when moving obstacles without tipping over. Of course, if necessary, the AGV's counterweight can be increased to provide sufficient assurance for the stability of obstacles.

[0035] In some cases, positioning devices can be installed on the transport equipment and connected to the internet. This allows the control system to obtain real-time location information for each transport device.

[0036] In some examples, the obstacle can be, but is not limited to, wall panels, dynamically deformable walls, etc. When the obstacle is a wall panel and the transport device is an AGV, please refer to... Figure 4 and Figure 5The images shown are, respectively, an exemplary side view and a top view of an AGV transport wall panel.

[0037] In some cases, the obstacles are of uniform, fixed size, which facilitates their handling and management.

[0038] In some examples, the transport device uses its own robotic arm to grab and transport the obstacle.

[0039] In some examples, the control system acquires the device location information in real time to perform dynamic layout calculations based on the obstacle layout information and the device location information.

[0040] In some examples, the control system performs layout calculations based on the obstacle layout information and the equipment location information, and then generates layout control commands according to the obtained optimal layout control method. This optimal layout control method can be the layout control method with the shortest path, the layout control method with the shortest control time, or a layout control method determined based on manually set priorities. This application does not specifically limit this method.

[0041] In some examples, the layout control instructions obtained after layout calculation may include layout control instructions for multiple handling devices. For example, handling device 1 controls obstacle 2 to target position 3, and handling device 2 controls obstacle 3 to target position 4. In the aforementioned layout calculation, an algorithm to avoid collisions between handling devices is often added, so that the generated layout control method can avoid collisions between handling devices.

[0042] In some examples, the obstacle layout corresponding to the obstacle layout information after the transport is completed can be square, rectangle, circle, trapezoid, triangle, etc. Specifically, if the obstacle layout after transport is circular, the obstacle can be a flexible wall panel or a plate chain. The transport equipment (or the robotic arm of the transport equipment) can be connected to preset connection points on both sides and in the middle of the flexible panel or plate chain. The control system controls the transport equipment (or the robotic arm of the transport equipment), causing the flexible panel or plate chain to deform, ultimately making the obstacle layout circular.

[0043] It is not difficult to see that, compared with related technologies, the simulation system based on semi-physical simulation provided in this application, after obtaining the device location information uploaded by the transport equipment and the obstacle layout information of the virtual scene from the control system, performs layout calculations based on the obstacle layout information and the device location information to obtain layout control instructions, and sends the layout control instructions to the transport equipment, so that the transport equipment transports the obstacles according to the layout control instructions. Thus, the site after transport can present an obstacle layout corresponding to the obstacle layout information. On the one hand, since the positions of the obstacles in this application embodiment are variable and flexibly set, the requirements for the site are reduced. On the other hand, since the obstacles can be automatically transported according to the obstacle layout information of the virtual scene to form the required obstacle layout of the site, the same site can meet the obstacle layout requirements of multiple virtual scenes. In other words, even if the building layout in the virtual scene changes, the obstacle layout corresponding to the actual site will be automatically adjusted to adapt to the change, thereby reducing the scene deployment cost.

[0044] In some embodiments of this application, the control system is further configured to, after receiving obstacle layout information of a virtual scene, match the obstacle layout information of the virtual scene with the current actual site information to obtain a matching result;

[0045] The control system is further configured to automatically divide the obstacle layout information of the virtual scene into several stages according to the matching result, perform layout calculations based on the obstacle layout information after being divided into several stages and the device position information, obtain layout control instructions, and send the layout control instructions to the handling equipment.

[0046] Combination Figure 2 As shown, in some examples, the matching of obstacle layout information in the virtual scene with the current actual site information can be based on area matching. For instance, if the matching result shows that the virtual scene's area is three times the size of the actual area, then the obstacle layout information in the virtual scene can be divided into three stages. The control system sequentially performs layout calculations based on the obstacle layout information of these three stages and the real-time position information of the equipment to obtain layout control commands, and then sends these commands to the transport equipment.

[0047] In some examples, the matching of obstacle layout information in the virtual scene with the current actual site information can be time-based. For instance, if the matching result shows that the virtual scene's terrain changed at the 3rd and 5th minute, the obstacle layout information can be divided into two stages. The control system then sequentially performs layout calculations based on the obstacle layout information from these two stages and the real-time location information of the equipment to obtain layout control commands, which are then sent to the transport equipment.

[0048] It is not difficult to see that, compared with related technologies, the simulation system based on semi-physical simulation provided in this application, after receiving obstacle layout information of the virtual scene, matches the obstacle layout information of the virtual scene with the current actual site information to obtain the matching result, and generates layout control instructions based on the matching result. In this way, while reducing the cost of scene deployment, the utilization rate of the current actual site can be further improved, and the requirements for the site are further reduced.

[0049] In some embodiments of this application, the system further includes a person recognition device;

[0050] The person recognition device is used to identify and obtain the location information of the user located in the venue, and send the user location information to the control system;

[0051] The control system is specifically used to receive obstacle layout information of the virtual scene, perform layout calculations based on the obstacle layout information of the virtual scene, the device location information and the user location information, obtain layout control instructions, and send the layout control instructions to the handling equipment.

[0052] Here, "user" refers to a player, and the number of users can be one or more.

[0053] In some examples, the person recognition device can be a positioning device installed on the user, or it can be a video surveillance device installed at an actual location that has the function of obtaining the user's location. No specific limitation is made here.

[0054] In some examples, users can choose which virtual scene they want to enter: a path leading to the left is virtual scene A, and a path leading to the right is virtual scene B. Such services, which aim to provide users with a better virtual simulation experience, can meet the above requirements by adopting the solution provided in the embodiments of this application.

[0055] Compared with related technologies, the simulation system based on semi-physical simulation provided in this application performs layout calculations based on obstacle layout information, device location information, and user location information in the virtual scene. Since the user location information is taken into account, higher quality layout control instructions can be obtained. Furthermore, since the user's location information may be an identifier for opening a new virtual scene, it can meet diverse virtual simulation service needs.

[0056] In some embodiments of this application, the control system is further configured to receive control commands for instructing changes to the display posture of the target obstacle;

[0057] The control system is specifically configured to, upon receiving a control command indicating a change in the display posture of a target obstacle, acquire the current position information and current display status information of the target obstacle, determine the adjustment method for adjusting the display posture of the target obstacle based on the current position information and current display status information of the target obstacle, and generate an adjustment command to send to the conveying device.

[0058] The transport device is specifically used to adjust the display posture of the obstacle according to the adjustment command.

[0059] In some examples, if the transport equipment and the target obstacle are rigidly connected based on structural components, the control system can calculate the current position and display status of the target obstacle by acquiring the current state of the transport equipment. If the transport equipment and the target obstacle are connected via a robotic arm, the control system can calculate the current display status of the target obstacle by acquiring the current state of the robotic arm mounted on the transport equipment, and determine the current position of the target obstacle based on the obstacle position information uploaded by the target obstacle. Alternatively, the control system can calculate the current display status of the target obstacle by acquiring the current state of the robotic arm mounted on the transport equipment. Furthermore, since the robotic arm and the transport equipment are connected in this case, the current position of the target obstacle can also be determined by acquiring the equipment position information of the transport equipment corresponding to the robotic arm.

[0060] In some examples, assuming the target obstacle is a wall panel, when the wall panel is displayed vertically perpendicular to the ground, it can be equivalent to a wall in the virtual scene; when the wall panel is displayed horizontally, it can be equivalent to a table in the virtual scene.

[0061] In practical applications, the adjustment commands sent to the conveying equipment can be rotation of ×× degrees, horizontal movement of ×× centimeters, vertical movement of ×× centimeters, etc., and no specific limitation is made here.

[0062] The aforementioned control commands for instructing changes to the display posture of the target obstacle can be entered into the control system by relevant operators during actual on-site operation based on the actual virtual simulation situation. Alternatively, they can be pre-entered into the control system as a dormant item by relevant operators, with a certain time or certain conditions set. Once the triggering conditions are met, the dormant item will be automatically triggered and activated, so that the control system actually receives the control commands for instructing changes to the display posture of the target obstacle.

[0063] Compared with related technologies, the simulation system based on semi-physical simulation provided in this application embodiment can bring users a more realistic and immersive experience because it can adjust the display posture of the obstacle according to the adjustment command.

[0064] In some embodiments of this application, the obstacle is equipped with a mechanism;

[0065] The aforementioned mechanism is used to receive trigger commands and synchronize the trigger commands to the control system;

[0066] The control system is used to combine the trigger command, perform layout calculations based on the obstacle layout information and the equipment position information, obtain layout control commands, and send the layout control commands to the handling equipment.

[0067] In some examples, the mechanism may be a handle, a button, etc.

[0068] For example, a wall panel held up by a robotic arm can be used as a closed "door". The "door" is equipped with a handle. If the user operates the handle on the "door", the "door" can be rotated open or slid open.

[0069] For example, a horizontal wall panel can be used as a "control panel." This "control panel" has two buttons. If the user presses one button, the "control panel" will automatically move horizontally upwards; if the user presses the other button, the "control panel" will automatically move horizontally downwards. In practical applications, the buttons can be labeled with corresponding arrows or other markings.

[0070] In the above example, the closed and open states of the door, as well as the switching modes of open-close and closed-open, can be pre-programmed in the control system. Then, when the user triggers the corresponding mechanism (issuing a trigger command), the control system can combine this pre-programmed information with the obstacle layout and equipment location information to perform layout calculations, obtain layout control commands, and send them to the transport equipment. This allows the transport equipment to move the obstacle according to the layout control commands.

[0071] Compared with related technologies, the simulation system based on semi-physical simulation provided in this application embodiment sets up mechanical devices on obstacles, and after the mechanical devices receive a trigger command, they synchronize the trigger command to the control system. This allows the control system to combine the trigger command with the obstacle layout information and the device position information to perform layout calculations, obtain layout control commands, and send the layout control commands to the transport equipment. In this way, it can further meet the diverse service needs of virtual simulation and provide users with a more varied and flexible virtual simulation experience.

[0072] This application also provides a simulation method based on hardware-in-the-loop simulation, applicable to systems as described in any of the above embodiments, such as... Figure 6 As shown, the method may include:

[0073] Step S101: Obtain the equipment location information of the handling equipment;

[0074] Step S102: Obtain obstacle layout information of the virtual scene, and perform layout calculation based on the obstacle layout information and the device position information to obtain layout control instructions;

[0075] Step S103: Control the transport equipment to transport the obstacle according to the layout control command, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information.

[0076] It is not difficult to see that the embodiments of this application are method embodiments corresponding to any one or more of the system embodiments described above. The methods described in the system embodiments are also applicable to the method embodiments. To avoid repetition, they will not be described again here.

[0077] Furthermore, this application also provides a simulation device based on hardware-in-the-loop simulation, the structure of which is as follows: Figure 7 As shown, the device includes a memory 11 for storing computer-readable instructions and a processor 12 for executing the computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the processor is triggered to execute the simulation method based on hardware-in-the-loop simulation.

[0078] The methods and / or embodiments in this application can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a processing unit, it performs the functions defined in the methods of this application.

[0079] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0080] In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0081] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0083] In another aspect, embodiments of this application also provide a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The aforementioned computer-readable medium carries one or more computer-readable instructions, which may be executed by a processor to implement the steps of the methods and / or technical solutions of the various embodiments of this application.

[0084] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0086] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0087] Furthermore, this application also provides a computer program stored in a computer device, which enables the computer device to execute the method for executing the control code.

[0088] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0089] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A simulation system based on hardware-in-the-loop simulation, characterized in that, The system includes a control system, handling equipment, and obstacles; The conveying device is used to send its device location information to the control system; The control system is used to receive obstacle layout information of the virtual scene, and after receiving the obstacle layout information of the virtual scene, it matches the obstacle layout information of the virtual scene with the current actual site information to obtain a matching result. The matching includes matching methods based on area ratio or based on virtual scene change time. The control system is further specifically used to automatically divide the obstacle layout information of the virtual scene into several stages according to the matching result, and perform layout calculations according to the obstacle layout information divided into several stages and the equipment position information in sequence to obtain layout control instructions, and send the layout control instructions to the handling equipment. The transport equipment is also used to transport the obstacles according to the layout control instructions, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information. When the building layout in the virtual scene changes, the obstacle layout in the actual site is automatically adjusted to adapt to the change, thereby reducing the scene deployment cost.

2. The system according to claim 1, characterized in that, The system also includes a person recognition device; The person recognition device is used to identify and obtain the location information of the user located in the venue, and send the user location information to the control system; The control system is specifically used to receive obstacle layout information of the virtual scene, perform layout calculations based on the obstacle layout information of the virtual scene, the device location information and the user location information, obtain layout control instructions, and send the layout control instructions to the handling equipment.

3. The system according to claim 1, characterized in that, The transport equipment and the obstacle are in one-to-one correspondence; The transport equipment is specifically used to transport obstacles corresponding to the transport equipment according to the layout control instructions, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information.

4. The system according to claim 1, characterized in that, One of the conveying devices corresponds to multiple of the obstacles; The obstacle is used to send its obstacle location information to the control system; The control system is specifically used to receive obstacle layout information of the virtual scene, and perform layout calculations based on the obstacle layout information of the virtual scene, the device location information, and the obstacle location information to obtain layout control instructions, and send the layout control instructions to the handling equipment. The layout control command includes instruction information for the obstacles that the handling equipment is to control.

5. The system according to claim 1, characterized in that, The control system is also configured to receive control commands for instructing changes to the display posture of the target obstacle; The control system is specifically configured to, upon receiving a control command indicating a change in the display posture of a target obstacle, acquire the current position information and current display status information of the target obstacle, determine the adjustment method for adjusting the display posture of the target obstacle based on the current position information and current display status information of the target obstacle, and generate an adjustment command to send to the conveying device. The transport device is specifically used to adjust the display posture of the obstacle according to the adjustment command.

6. The system according to claim 1, characterized in that, The obstacle is equipped with mechanical devices; The aforementioned mechanism is used to receive trigger commands and synchronize the trigger commands to the control system; The control system is used to combine the trigger command, perform layout calculations based on the obstacle layout information and the equipment position information, obtain layout control commands, and send the layout control commands to the handling equipment.

7. A simulation method based on hardware-in-the-loop simulation, characterized in that, Applied to the system as described in any one of claims 1 to 6, the method comprises: Obtain the device location information of the conveying equipment; Obtain obstacle layout information of the virtual scene; after receiving the obstacle layout information of the virtual scene, match the obstacle layout information of the virtual scene with the current actual site information to obtain the matching result. The matching includes matching methods based on area ratio or based on virtual scene change time. Based on the matching results, the obstacle layout information of the virtual scene is automatically divided into several stages, and layout calculations are performed sequentially based on the obstacle layout information after being divided into several stages and the device location information to obtain layout control instructions. The transport equipment is controlled to transport the obstacles according to the layout control instructions, so that the site after the transport is completed presents an obstacle layout corresponding to the obstacle layout information. When the building layout in the virtual scene changes, the obstacle layout in the actual site is automatically adjusted to adapt to the change, thereby reducing the scene deployment cost.

8. A simulation device based on hardware-in-the-loop simulation, characterized in that, The device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the method as described in claim 7.

9. A computer-readable medium having stored thereon computer program instructions that can be executed by a processor to implement the method of claim 7.

Images