Terminal equipment control method, terminal linkage system and storage medium

Abstract

The invention provides a terminal equipment control method, a terminal linkage system and a storage medium, and the method comprises the steps: receiving real-time video data sent by video collection equipment, the real-time video data being video data comprising images of terminal equipment and a user; receiving high-precision positioning data sent by the Beidou positioning receiver, wherein the high-precision positioning data comprises equipment positioning data of the terminal equipment and user positioning data of the user; integrating the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generating a control instruction of the terminal equipment at least according to the comprehensive information data; and sending the control instruction to the terminal equipment, so that the terminal equipment executes the target operation according to the control instruction. According to the method, real-time accurate positioning can be realized, so that the safety control of a working site is comprehensively improved, and the safety of personnel in the working process is ensured.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a control method for a terminal device, a terminal linkage system, a computer-readable storage medium, and an electronic device. Background Art

[0002] At present, the search for target personnel at the work site mainly relies on the combination of traditional surveillance ball and video target detection. However, this method has obvious limitations in practical applications. Due to the complex and changeable environment of the work site, workers often need to work in different locations, and their movement trajectories are difficult to predict. Although the traditional surveillance ball plays an indispensable role in the field of monitoring, its monitoring range has obvious limitations. This type of equipment can usually only cover a limited area, and it is often unable to effectively capture targets beyond its set range. At the same time, its monitoring angle is relatively fixed, and it is difficult to adapt to the changing environment and monitoring needs. This makes the traditional surveillance ball seem powerless when facing some complex scenes or large-scale monitoring tasks. Therefore, although it can play an important role in certain specific occasions, when pursuing comprehensive and blind-angle monitoring effects, the traditional surveillance ball obviously cannot meet the higher requirements of modern monitoring technology. In this case, due to its fixed viewing angle and shooting range, it is difficult for the traditional surveillance ball to fully cover all possible work areas. Therefore, it is very easy for workers to appear outside the shooting range of the surveillance ball, resulting in the inability to effectively capture and detect the target personnel.

[0003] At the same time, although the video target detection method can identify the target person in the video picture, its effect is also affected by many factors such as picture quality and target person characteristics. If the operator appears outside the picture, or the picture quality is poor due to angle, light and other issues, it is difficult for the video target detection method to accurately identify the target person.

[0004] Therefore, the traditional method of surveillance ball + video target detection has obvious shortcomings when searching for target personnel at the work site. Summary of the invention

[0005] The main purpose of the present application is to provide a control method for a terminal device, a terminal linkage system, a computer-readable storage medium and an electronic device, so as to at least solve the problem that the detection method for finding the target personnel at the work site in the prior art cannot cover the entire work area and the positioning effect is poor.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a control method for a terminal device is provided, comprising: receiving real-time video data sent by the video acquisition device, the real-time video data being video data including images of the terminal device and a user; receiving high-precision positioning data sent by the Beidou positioning receiver, the high-precision positioning data including device positioning data of the terminal device and user positioning data of the user; integrating the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generating a control instruction for the terminal device at least based on the comprehensive information data; sending the control instruction to the terminal device so that the terminal device performs the target operation according to the control instruction.

[0007] Optionally, the real-time video data and the high-precision positioning data are integrated to obtain comprehensive information data, including: preprocessing the real-time video data and the high-precision positioning data respectively to obtain processed video data and processed positioning data; using a data fusion algorithm to synchronously integrate the processed video data and the processed positioning data to obtain the comprehensive information data.

[0008] Optionally, the real-time video data and the high-precision positioning data are preprocessed respectively to obtain processed video data and processed positioning data, including: decoding the real-time video data to obtain decoded video data, wherein the decoded video data is a digital signal recognizable by the central processing unit; using an image recognition and processing algorithm to extract video key information from the decoded video data to obtain the processed video data, wherein the video key information includes at least one of the following: the user's position, the position, speed and movement direction of the target object, and the image recognition and processing algorithm includes at least a target detection algorithm, a target tracking algorithm and a target recognition algorithm, and the image recognition and processing algorithm is obtained based on deep learning technology or machine learning technology.

[0009] Optionally, the real-time video data and the high-precision positioning data are preprocessed respectively to obtain processed video data and processed positioning data, including: parsing the high-precision positioning data, extracting the longitude data and latitude data of the terminal device and the longitude data and latitude data of the user, and obtaining the processed positioning data, wherein the accuracy of the processed positioning data is at the centimeter level.

[0010] Optionally, the control instructions of the terminal device are generated at least based on the comprehensive information data, including: generating a driving path of the terminal device based on the comprehensive information data, where there are no users and target objects on the driving path of the terminal device; determining an operation strategy of the terminal device within a target time period based on the driving path of the terminal device, where the operation strategy includes at least an operation direction and an operation speed of the terminal device at each moment within the target time period; receiving a user instruction, where the user instruction is generated based on information input by the user into a user interface; and determining the control instructions of the terminal device based on the user instruction, the driving path of the terminal device, and the operation strategy of the terminal device within the target time period.

[0011] Optionally, the terminal device is a vehicle, and the comprehensive information data includes status data of the target object. The control instruction of the terminal device is generated according to the comprehensive information data, including: when the target object is an indicator light and the status data of the target object is color data of the indicator light, the control instruction of the terminal device is determined according to the color data of the indicator light, wherein when the color of the indicator light is red, the control instruction of the terminal device is a stop instruction; when the target object is an obstacle and the status data of the target object is position data of the obstacle, the control instruction of the terminal device is determined according to the position data of the obstacle to avoid a collision between the terminal device and the obstacle, wherein the control instruction of the terminal device includes the running direction and running speed of the terminal device.

[0012] Optionally, after receiving the high-precision positioning data of the terminal device sent by the Beidou positioning receiver, the method further includes: determining whether there is a risk of collision between the terminal device and the user based on the real-time video data and the high-precision positioning data; when there is a risk of collision between the terminal device and the user, automatically adjusting the operating status of the terminal device and / or sending an alarm message.

[0013] According to another aspect of the present application, a terminal linkage system is provided, comprising: a central processing unit, used to execute any one of the control methods for the terminal device; a video acquisition device, used to acquire video data including images of the terminal device and the user, the video acquisition device being communicatively connected to the central processing unit; a Beidou positioning receiver, used to acquire device positioning data of the terminal device and user positioning data of the user, the Beidou positioning receiver being communicatively connected to the central processing unit; and a terminal device, being communicatively connected to the central processing unit.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the terminal device control methods.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a control method for executing any one of the terminal devices described.

[0016] Applying the technical solution of the present application, the control method of the above terminal device first receives the real-time video data sent by the video acquisition device, and the real-time video data is the video data including the images of the terminal device and the user; then receives the high-precision positioning data sent by the Beidou positioning receiver, and the high-precision positioning data includes the device positioning data of the terminal device and the user positioning data of the user; then integrates the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generates the control instructions of the terminal device at least according to the comprehensive information data; finally sends the control instructions to the terminal device, so that the terminal device performs the target operation according to the control instructions. The method can obtain the real-time image and precise positioning of the device, realize the linkage of video and positioning technology, so as to accurately locate the position and real-time dynamics of the target personnel, and can accurately integrate the video data and positioning data and output the control instructions of the terminal device, realize the linkage control of the terminal positioning and the video device in three-dimensional space, solve the problem that the detection method of finding the target personnel at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, improve the flexibility and accuracy of video monitoring, thereby comprehensively improving the safety management and control of the operation site, and ensuring the safety of personnel during the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for controlling a terminal device provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of a process flow of a method for controlling a terminal device provided according to an embodiment of the present application is shown;

[0020] Figure 3A schematic diagram of a process of front-end ball control authority provided according to an embodiment of the present application is shown;

[0021] Figure 4 A diagram showing an example of an application scenario of a method for controlling a terminal device provided according to an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram showing the principle of Bluetooth transmission of a front-end control ball provided according to an embodiment of the present application is shown;

[0023] Figure 6 A schematic diagram of the structure of a terminal linkage system provided according to an embodiment of the present application is shown;

[0024] Figure 7 A structural block diagram of a central processing unit provided according to an embodiment of the present application is shown.

[0025] The above drawings include the following reference numerals:

[0026] 01. Central processing unit; 02. Video acquisition device; 03. Beidou positioning receiver; 04. Terminal device; 102. Processor; 104. Memory; 106. Transmission device; 108. Input and output devices. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] As introduced in the background technology, the search for target personnel in the prior art relies on a method that combines traditional surveillance balls with video target detection. However, the traditional surveillance ball monitoring method has the limitations of limited monitoring coverage and relatively fixed monitoring angles, which results in the target personnel being unable to be effectively captured and monitored. At the same time, the detection effect of the video target detection method is unstable due to various factors, which affects the accuracy of target personnel identification. In order to solve the problem that the detection method for searching target personnel at the work site in the prior art cannot cover the entire work area and has poor positioning effect, the embodiments of the present application provide a control method for a terminal device, a terminal linkage system, a computer-readable storage medium and an electronic device.

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a terminal device control method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the control method of the terminal device in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] In this embodiment, a control method for a terminal device running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] A control method for a terminal device is applied to a central processing unit in a terminal linkage system. The terminal linkage system also includes a video acquisition device, a Beidou positioning receiver and a terminal device. The central processing unit is respectively communicatively connected with the video acquisition device, the Beidou positioning receiver and the terminal device. Figure 2 1 is a flow chart of a control method of a terminal device according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0036] Step S201, receiving real-time video data sent by the video acquisition device, wherein the real-time video data is video data including images of the terminal device and the user;

[0037] Specifically, the video acquisition device, i.e., the high-definition camera, is directly connected to the central processing unit through a physical cable such as a network cable, or is transmitted to the central processing unit through wireless transmission technology such as Wi-Fi. One or more high-definition cameras can capture visual information in real time, convert it into a digital video signal, and transmit it to the central processing unit, so as to realize efficient and accurate transmission of video data of terminal devices and user images, and provide strong support for subsequent linkage control.

[0038] The central processing unit can quickly respond to the real-time received data stream, generate and execute control instructions, and ensure the real-time and accuracy of the system. The video acquisition device can be one or more high-definition cameras to capture and transmit video signals in real time.

[0039] Among them, one implementation case of this application uses a surveillance ball as a video acquisition device, such as Figure 3 As shown, turn on the video acquisition device of the control ball, wait for the input of the user name of the control ball machine, determine whether the user exists, if not, return to the previous level, if the user name exists, wait for the input of the password of the control ball machine, then determine whether the password exists, if not, return, if it exists, enter the selection panel. The selection panel is divided into single, group and query record. If you choose single, select the corresponding machine id to log in, determine whether there is permission, if yes, complete the login, otherwise it will return to the previous level; if you choose group, enter the id of the corresponding group, determine whether there is permission, if yes, wait for the input of the corresponding command, then complete the recording of the corresponding information, otherwise it will return to the previous level; select query record to directly query the corresponding result.

[0040] Step S202, receiving high-precision positioning data sent by the Beidou positioning receiver, the high-precision positioning data including the device positioning data of the terminal device and the user positioning data of the user;

[0041] Specifically, the Beidou positioning receiver receives Beidou satellite signals through a built-in antenna, and parses the positioning information through the internal processing circuit. The above positioning information is longitude and latitude data in digital form. These data are directly transmitted to the central processing unit through a physical interface such as USB or a wireless connection such as GPS, realizing efficient and accurate transmission of device positioning data and user positioning data, providing strong support for subsequent linkage control. After that, the geographical location of the terminal device is determined by processing these signals, and positioning data with at least meter-level accuracy can be provided.

[0042] Among them, the above-mentioned users are operating personnel in the operating scenario, and the above-mentioned terminal devices are operating equipment in the operating scenario.

[0043] Step S203, integrating the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generating a control instruction for the terminal device at least based on the comprehensive information data;

[0044] The above-mentioned real-time video data and the above-mentioned high-precision positioning data are integrated to obtain comprehensive information data, including the following steps:

[0045] Step S301, preprocessing the real-time video data and the high-precision positioning data to obtain processed video data and processed positioning data;

[0046] Specifically, the central processing unit receives the real-time video data stream from the video acquisition device through a physical connection or a wireless connection. At the same time, the central processing unit also receives the positioning data stream from the Beidou positioning receiver. These data streams contain accurate geographic location information. The central processing unit decodes the received video data stream and parses the Beidou positioning data stream to extract the digital signal and positioning information respectively. The central processing unit can respond quickly according to the real-time received data stream and complete the automatic processing of the data.

[0047] The real-time video data and the high-precision positioning data are preprocessed to obtain processed video data and processed positioning data, respectively, including the following steps:

[0048] Step S3011, decoding the real-time video data to obtain decoded video data, wherein the decoded video data is a digital signal recognizable by the central processing unit;

[0049] Step S3012, using an image recognition and processing algorithm to extract video key information from the above-mentioned decoded video data to obtain the above-mentioned processed video data, the above-mentioned video key information includes at least one of the following: the user's position, the position, speed and movement direction of the target object, the above-mentioned image recognition and processing algorithm includes at least a target detection algorithm, a target tracking algorithm and a target recognition algorithm, and the above-mentioned image recognition and processing algorithm is obtained based on deep learning technology or machine learning technology.

[0050] Specifically, the central processing unit first decodes the received video data stream and converts it into a processable digital signal, and then decodes the Beidou positioning data stream to extract key positioning information such as longitude and latitude. The extracted valid data can provide strong support for subsequent analysis, thereby realizing the integration of different source data and reflecting the linearization of data flow. Through the preset logical rules and algorithms in the central processing unit, the use of deep learning, machine learning and other technologies can improve the accuracy and efficiency of recognition.

[0051] Among them, the above-mentioned real-time video data and the above-mentioned high-precision positioning data are pre-processed respectively to obtain processed video data and processed positioning data, including: parsing the above-mentioned high-precision positioning data, extracting the longitude data and latitude data of the above-mentioned terminal device and the longitude data and latitude data of the above-mentioned user, and obtaining the above-mentioned processed positioning data, and the accuracy of the above-mentioned processed positioning data is at the centimeter level.

[0052] Specifically, by preprocessing real-time video data and high-precision positioning data, more accurate positioning information can be obtained. The accuracy of the positioning data after the above processing can reach centimeter-level accuracy, providing an accurate basis for subsequent navigation and analysis.

[0053] One implementation example Figure 4 As shown in the figure, in order to ensure the safety of medium- and high-risk workers, to quickly find, track, and identify the above-mentioned workers, and to send control instructions through terminal devices, we place a control ball at a suitable location on the job site. The control ball is a video acquisition device and a Beidou positioning receiver. By continuously identifying and locating workers and / or helmets, we obtain the video data and positioning information at that time. At the same time, the gateway serves as a communication interface, which is configured to send video data and positioning data to a remote server or user device, and receive control signals from a remote server or user device. When these positioning information are sent to the system, advanced algorithm technology is used to perform accurate calculations based on the sent positioning information and the positioning data of the control ball. Through these calculations, the relative position relationship between the control ball and the position of the personnel is obtained, including detailed information such as distance and angle. After that, the intelligent terminal will be used to accurately control the pan / tilt of the control ball. According to the calculated relative position relationship, the intelligent terminal will control the pan / tilt of the control ball to rotate the corresponding angle and adjust the focal length to ensure that the camera can accurately aim at the target personnel. This process realizes the linkage control of terminal positioning and video equipment in three-dimensional space, greatly improving the flexibility and accuracy of video surveillance. In this way, we can quickly find, track and identify medium and high-risk operators. No matter where they are in the work site, we can quickly find their location and monitor their dynamics in real time through the linkage of video and positioning technology. This not only solves the monitoring blind spot problem of traditional video surveillance methods, but also makes target search more convenient and efficient.

[0054] Step S302: Using a data fusion algorithm, synchronously integrate the processed video data and the processed positioning data to obtain the comprehensive information data.

[0055] Specifically, the central processing unit can integrate the processed video data and positioning data from different sources through steps such as time synchronization and data format conversion, so as to ensure the accuracy and consistency of the information.

[0056] The step of generating the control instruction of the terminal device at least according to the comprehensive information data comprises the following steps:

[0057] Step S401, generating a driving path of the terminal device according to the comprehensive information data, where there is no user or target object on the driving path of the terminal device;

[0058] Step S402, determining an operation strategy of the terminal device within a target time period according to the driving path of the terminal device, wherein the operation strategy at least includes an operation direction and an operation speed of the terminal device at each time within the target time period;

[0059] Step S403, receiving a user instruction, where the user instruction is generated based on the information input into the user interface by the user;

[0060] Step S404: determining a control instruction for the terminal device according to the user instruction, the driving path of the terminal device, and the operation strategy of the terminal device within the target time period.

[0061] Specifically, by integrating a variety of comprehensive information, such as the running direction and running speed at each moment within the target time period to generate the driving path of the terminal device, and determining the optimal driving strategy for the above terminal device within a specific time, it can be ensured that the terminal device can operate according to the optimal path and optimal strategy within the predetermined target time period, thereby improving operating efficiency and accuracy, improving the overall scheduling and navigation capabilities of the system, and improving the rationality and safety of the navigation system.

[0062] The user instructions are input into the user interface by non-operating personnel, and the user interface may be another controller device.

[0063] The generation of the driving path of the terminal device is based on path planning and navigation algorithms. The optimal path is planned for the terminal device through positioning data, combined with map information and obstacle detection. This method takes into account factors such as real-time traffic information and road conditions, and can ensure safe and efficient navigation. Then, according to the identified target, current location, preset target and other factors, a decision logic is formulated, such as stopping when encountering a red light, or choosing to detour when an obstacle is detected. Then, through instruction coding and transmission protocol, the decision is converted into specific control instructions, such as forward, backward, left turn, right turn, etc. The above instructions need to be sent to the remote terminal device according to specific coding and transmission protocols to ensure accurate communication and execution of the instructions. Finally, based on the conditional judgment and instruction matching mechanism, the central processing unit performs conditional judgment on the integrated information. For example, if the video data shows that there is an obstacle ahead and the positioning data shows that the vehicle is approaching a specific area, the central processing unit will match the preset logic rules according to these conditions. After the match is successful, the operation to be performed will be determined, such as deceleration, steering or parking, and specific control instructions will be generated. The above instructions directly correspond to the executable operations of the remote terminal device.

[0064] In some embodiments, the terminal device is a vehicle, the comprehensive information data includes state data of the target object, and generating the control instruction of the terminal device according to the comprehensive information data includes the following steps:

[0065] Step S501, when the target object is an indicator light and the state data of the target object is color data of the indicator light, determining a control instruction of the terminal device according to the color data of the indicator light, wherein when the color of the indicator light is red, the control instruction of the terminal device is a stop instruction;

[0066] Step S502, when the target object is an obstacle and the status data of the target object is the position data of the obstacle, determine the control instruction of the terminal device according to the position data of the obstacle to avoid a collision between the terminal device and the obstacle, wherein the control instruction of the terminal device includes the running direction and speed of the terminal device.

[0067] Specifically, based on the integrated comprehensive information, there is a safety verification mechanism in the process of the central processing unit generating corresponding control instructions according to the preset logic or algorithm. Safety verification is performed before the instructions are sent to check the rationality and safety of the instructions to prevent potential dangers caused by misidentification or data transmission errors. This improves the adaptability of terminal devices in complex environments and driving safety.

[0068] Step S204: sending the control instruction to the terminal device, so that the terminal device performs the target operation according to the control instruction.

[0069] Specifically, before sending the instruction, the central processing unit may conduct a quick verification process to ensure that the generated instruction is reasonable and safe. After the verification is passed, the instruction will be output and ready to be sent to the remote terminal device for execution; the generated control instruction is sent to the remote terminal device through a physical connection or wireless transmission. After receiving the instruction, the remote terminal device performs the corresponding operation, thereby realizing the precise control of the terminal device by the central processing unit. This method is highly automated and can ensure that the system responds quickly and correctly in a complex environment, generates and executes control instructions, and ensures the real-time and accuracy of the system. At the same time, the data flow process of this method is linear, the process is simple and easy to understand and implement, which facilitates error location and repair, improves processing efficiency, and is easy to expand and maintain.

[0070] Wherein, after receiving the high-precision positioning data of the terminal device sent by the Beidou positioning receiver, the method further includes the following steps:

[0071] Step S601, determining whether there is a collision risk between the terminal device and the user according to the real-time video data and the high-precision positioning data;

[0072] Step S602: When there is the above-mentioned collision risk between the above-mentioned terminal device and the above-mentioned user, the operating state of the above-mentioned terminal device is automatically adjusted and / or an alarm message is sent.

[0073] Specifically, the central processing unit includes an anomaly detection module that can identify anomalies in video data and positioning data and trigger a preset emergency response program, which may include automatically adjusting the operating state of the terminal device or sending an alarm message to the user device. In this way, by detecting anomalies and automatically adjusting the operating state of the terminal device and / or sending an alarm message, the risk of collision can be avoided, the stability and safety of the system can be ensured, and the risk and cost of system operation can be effectively reduced.

[0074] In one embodiment, Figure 5As shown, the control method of the terminal device is applied to the vehicle entry gate system. The system includes six modules: Bluetooth, control ball, switch, gate, computer and wired button. The modules are connected by network cables. When the vehicle enters the operation scene, the control ball acts as a video acquisition device and Beidou positioning receiver to collect real-time video data and high-precision positioning data of the entering vehicle. The central processing unit, i.e., the switch, decodes and processes the video data and positioning data respectively, and integrates and organizes the data to obtain comprehensive information data. At least the control instructions of the terminal device are generated according to the comprehensive information data, i.e., the control instructions of whether to allow the vehicle to pass are generated for the gate in this operation scene. At the same time, the system is also provided with a wired button, which can realize the function of staff assisting in controlling the gate.

[0075] The control method of the terminal device of the present application first receives the real-time video data sent by the video acquisition device, and the real-time video data is the video data including the images of the terminal device and the user; then receives the high-precision positioning data sent by the Beidou positioning receiver, and the high-precision positioning data includes the device positioning data of the terminal device and the user positioning data of the user; then integrates the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generates the control instructions of the terminal device at least according to the comprehensive information data; finally sends the control instructions to the terminal device, so that the terminal device performs the target operation according to the control instructions. The method can obtain the real-time image and precise positioning of the device, realize the linkage of video and positioning technology, so as to accurately locate the position and real-time dynamics of the target personnel, and can accurately integrate the video data and positioning data and output the control instructions of the terminal device, realize the linkage control of the terminal positioning and the video device in three-dimensional space, solve the problem that the detection method of finding the target personnel at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, improve the flexibility and accuracy of video monitoring, thereby comprehensively improving the safety management and control of the operation site, and ensuring the safety of personnel during the operation.

[0076] The control method of the above terminal equipment deeply integrates Beidou high-precision positioning technology and real-time video technology. It can not only ensure the precise positioning of the equipment through Beidou high-precision positioning technology, but also provide real-time images of the equipment through real-time video technology, so that users can understand the operation of the equipment and the surrounding environment in real time and intuitively. This method has significant advantages such as accurate positioning, rapid response, and easy operation. It is suitable for a variety of application scenarios such as intelligent traffic management, logistics and transportation tracking, and security monitoring. Through this terminal linkage system, users can manage and control remote devices more conveniently. At the same time, the intelligence and automation level of the system can greatly improve work efficiency and safety.

[0077] In some embodiments, the control method of the terminal device can be applied to an intelligent traffic navigation system. First, the system uses data fusion technology to integrate the real-time positioning data from the Beidou positioning receiver and the traffic status video data obtained by the video acquisition device. This step ensures the accurate synchronization of the two data through the data fusion algorithm, providing an accurate information basis for subsequent path planning and navigation; then, based on the above fused and sorted data, the central processing unit uses Dijkstra, A* and other search algorithms to perform real-time vehicle path planning. The above search algorithms can quickly find the optimal path in a static or dynamic environment, which can improve travel efficiency and save time; finally, according to the path planning results, the central processing unit generates specific navigation instructions, such as "turn left 500 meters ahead", "go straight through the next intersection", etc. The above navigation instructions are sent to the vehicle navigation device or mobile phone APP through wireless transmission, which can guide the driver to reach the destination safely and efficiently.

[0078] In some embodiments, the control method of the terminal device is also applied to the control system of an autonomous driving vehicle. The video acquisition device continuously captures the visual information around the vehicle to perceive the environment in which the vehicle is located, and combines the deep learning algorithm to detect and identify targets, including pedestrians, vehicles, traffic signs, etc. At the same time, the Beidou positioning receiver can provide accurate location information, which is combined with the high-precision map to match the vehicle position, so as to achieve accurate positioning of the vehicle and real-time monitoring of the driving route.

[0079] In another embodiment, the intelligent security monitoring system adopts the control method of the above terminal device, and monitors the key areas in real time through video acquisition equipment such as high-definition cameras to capture any abnormal dynamics. Combined with image recognition technology, the system can automatically detect abnormal events, such as intrusion, fire, etc.; once an abnormal situation occurs, the Beidou positioning receiver can quickly track and accurately locate the location of the event; the central processing unit automatically generates an alarm instruction based on video analysis and positioning information, and sends it to the security personnel's mobile device through a wireless network, so as to respond to and handle abnormal situations in a timely manner.

[0080] In another embodiment, the control method of the terminal device is applied to the autonomous flight system of a drone. The video acquisition device provides real-time visual information for the drone. Through image processing and recognition technology, the drone can identify key information such as ground signs and obstacles to achieve visual navigation; the Beidou positioning receiver provides centimeter-level precision positioning services for the drone, ensuring that the drone can accurately navigate in complex environments; the central processing unit combines visual information and positioning data, and generates flight instructions such as climb, dive, hover, etc. according to preset flight logic and algorithms. These instructions are executed through the drone's control system to achieve autonomous flight and precise operation of the drone.

[0081] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0082] The embodiment of the present application also provides a control system of a terminal device, Figure 6 FIG. 1 shows a schematic diagram of the structure of a terminal linkage system. Figure 6 As shown, the control system of the terminal device includes: a central processing unit 01, which is used to execute any control method of the terminal device; a video acquisition device 02, which is used to acquire video data including images of the terminal device 04 and the user, and the above-mentioned video acquisition device 02 is communicatively connected to the above-mentioned central processing unit 01; a Beidou positioning receiver 03, which is used to acquire the device positioning data of the above-mentioned terminal device 04 and the user positioning data of the above-mentioned user, and the above-mentioned Beidou positioning receiver 03 is communicatively connected to the above-mentioned central processing unit 01; the terminal device 04 is communicatively connected to the above-mentioned central processing unit 01.

[0083] The central processing unit includes a data processing module, which is used to analyze and interpret video data and positioning data, and a control instruction generation module, which generates control instructions based on the analysis results of the data processing module, such as a switch in some embodiments; the video acquisition device is used to capture real-time video data, and most systems use high-definition cameras for real-time video recording; the Beidou positioning receiver is used as a unit in the system to receive Beidou satellite signals and output high-precision positioning data. Data is transmitted between the acquisition device, Beidou positioning receiver and central processing unit through physical connection or wireless connection; the video acquisition device is directly connected to the central processing unit through a physical cable such as a network cable, or transmits data to the central processing unit through wireless transmission technology; the camera captures visual information in real time and converts it into digital video signals, which are transmitted to the central processing unit; the Beidou positioning receiver receives Beidou satellite signals through a built-in antenna, and parses the positioning information through the internal processing circuit. The information is longitude and latitude data in digital form; these data are transmitted to the central processing unit through a physical interface; the central processing unit receives the video data stream from the video acquisition device and the positioning data stream from the Beidou positioning receiver; the central processing unit decodes, analyzes and processes these two types of data, and integrates them to form comprehensive information; based on this information, the central processing unit generates control instructions, which are sent to the remote terminal device through physical connection or wireless transmission to achieve control.

[0084] The control system of the above-mentioned terminal device of the present application includes: a video acquisition device for capturing real-time video data; a Beidou positioning receiver for receiving Beidou satellite signals and outputting high-precision positioning data; a central processing unit for receiving and processing video data from the video acquisition device and positioning data from the Beidou positioning receiver, and generating control instructions based on these data to control the remote terminal device. The system can obtain real-time images and precise positioning of the device, realize the linkage of video and positioning technology, thereby accurately locating the position and real-time dynamics of the target personnel, and can accurately integrate the video data and positioning data and output the control instructions of the terminal device, realize the linkage control of terminal positioning and video equipment in three-dimensional space, solve the problem that the detection method for finding the target personnel at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, improve the flexibility and accuracy of video monitoring, and improve the response speed and reliability of the overall system, thereby comprehensively improving the safety management and control of the operation site and ensuring the safety of personnel during the operation.

[0085] The embodiment of the present application also provides a central processing unit. It should be noted that the central processing unit of the embodiment of the present application can be used to execute the control method for terminal equipment provided by the embodiment of the present application. The central processing unit is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the central processing unit described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0086] The central processing unit provided in the embodiment of the present application is introduced below.

[0087] Figure 7 is a schematic diagram of a central processing unit according to an embodiment of the present application. The central processing unit is in a terminal linkage system, the terminal linkage system further comprising a video acquisition device, a Beidou positioning receiver and a terminal device, the central processing unit is respectively connected to the video acquisition device, the Beidou positioning receiver and the terminal device for communication, such as Figure 7As shown, the central processing unit includes a first receiving unit 10, a second receiving unit 20, a processing unit 30 and a control unit 40, the first receiving unit 10 is used to receive the real-time video data sent by the above-mentioned video acquisition device, and the above-mentioned real-time video data is video data including images of the terminal device and the user; the second receiving unit 20 is used to receive the high-precision positioning data sent by the above-mentioned Beidou positioning receiver, and the above-mentioned high-precision positioning data includes the device positioning data of the above-mentioned terminal device and the user positioning data of the user; the processing unit 30 integrates the above-mentioned real-time video data and the above-mentioned high-precision positioning data to obtain comprehensive information data, and generates the control instructions of the above-mentioned terminal device at least according to the above-mentioned comprehensive information data; the control unit 40 is used to send the above-mentioned control instructions to the above-mentioned terminal device, so that the above-mentioned terminal device performs the target operation according to the above-mentioned control instructions.

[0088] The above-mentioned central processing unit of the present application includes a first receiving unit, a second receiving unit, a processing unit and a control unit, wherein the first receiving unit is used to receive real-time video data sent by a video acquisition device, and the real-time video data is video data including images of a terminal device and a user; the second receiving unit is used to receive high-precision positioning data sent by a Beidou positioning receiver, and the high-precision positioning data includes device positioning data of the terminal device and user positioning data of the user; the processing unit is used to integrate the real-time video data and the high-precision positioning data to obtain comprehensive information data, and at least generate a control instruction of the terminal device according to the comprehensive information data; the control unit is used to send the control instruction to the terminal device so that the terminal device performs the target operation according to the control instruction. The central processing unit can obtain the real-time image and precise positioning of the device, realize the linkage of video and positioning technology, thereby accurately locating the position and real-time dynamics of the target person, and can accurately integrate the video data and positioning data and output the control instruction of the terminal device, realize the linkage control of the terminal positioning and the video device in three-dimensional space, solve the problem that the detection method of finding the target person at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, improve the flexibility and accuracy of video monitoring, thereby comprehensively improving the safety management and control of the operation site, and ensuring the safety of personnel during the operation process.

[0089] In some examples, the processing unit includes a first processing module and a second processing module, wherein the first processing module is used to pre-process the above-mentioned real-time video data and the above-mentioned high-precision positioning data respectively to obtain processed video data and processed positioning data; the second processing module is used to synchronously integrate the above-mentioned processed video data and the above-mentioned processed positioning data using a data fusion algorithm to obtain the above-mentioned comprehensive information data. In this way, accurate geographic location information can be provided, which provides an important basis for the subsequent generation of accurate control instructions.

[0090] In some examples, the first processing module includes a first processing submodule and a second processing submodule, the first processing submodule is used to decode the above real-time video data to obtain decoded video data, the decoded video data is a digital signal recognizable by the above central processing unit; the second processing submodule is used to extract video key information from the above decoded video data using an image recognition and processing algorithm to obtain the above processed video data, the above video key information includes at least one of the following: the user's position, the position, speed and movement direction of the target object, the above image recognition and processing algorithm includes at least a target detection algorithm, a target tracking algorithm and a target recognition algorithm, the above image recognition and processing algorithm is obtained based on deep learning technology or machine learning technology. This can ensure the accuracy and consistency of information and improve the accuracy and efficiency of recognition.

[0091] In some examples, the first processing module further includes a third processing submodule, which is used to parse the high-precision positioning data, extract the longitude and latitude data of the terminal device and the longitude and latitude data of the user, and obtain the processed positioning data, the accuracy of which is at the centimeter level. In this way, more accurate positioning information can be obtained, and the accuracy can reach the centimeter level, providing an accurate basis for subsequent navigation and analysis.

[0092] In some examples, the processing unit includes a third processing module, a fourth processing module, a fifth processing module and a sixth processing module, wherein the third processing module is used to generate a driving path of the terminal device according to the comprehensive information data, and there is no user or target object on the driving path of the terminal device; the fourth processing module is used to determine the operation strategy of the terminal device within the target time period according to the driving path of the terminal device, and the operation strategy at least includes the operation direction and the operation speed of the terminal device at each moment within the target time period; the fifth processing module is used to receive user instructions, and the user instructions are generated according to the information input into the user interface by the user; the sixth processing module is used to determine the control instructions of the terminal device according to the user instructions, the driving path of the terminal device and the operation strategy of the terminal device within the target time period. In this way, it can be ensured that the terminal device can operate according to the optimal path and optimal strategy within the predetermined target time period, improve the operation efficiency and accuracy, improve the overall scheduling and navigation capabilities of the system, and improve the rationality and safety of the navigation system.

[0093] In some examples, the sixth processing module includes a fourth processing submodule and a fifth processing submodule, the fourth processing submodule is used to determine the control instruction of the terminal device according to the color data of the indicator light when the target object is an indicator light and the state data of the target object is the color data of the indicator light, wherein when the color of the indicator light is red, the control instruction of the terminal device is a parking instruction; the fifth processing submodule is used to determine the control instruction of the terminal device according to the position data of the obstacle when the target object is an obstacle and the state data of the target object is the position data of the obstacle, so as to avoid the terminal device from colliding with the obstacle, wherein the control instruction of the terminal device includes the running direction and running speed of the terminal device. This can improve the adaptability of the terminal device in complex environments and the driving safety.

[0094] In some examples, the control unit includes a first control module and a second control module, wherein the first control module is used to determine whether there is a collision risk between the terminal device and the user based on the real-time video data and the high-precision positioning data; and the second control module is used to automatically adjust the operating state of the terminal device and / or send an alarm message when there is a collision risk between the terminal device and the user. This can avoid collision risks, ensure the stability and safety of the system, and effectively reduce the risks and costs of system operation.

[0095] The terminal linkage system includes a processor and a memory, and the receiving unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0096] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem that the detection method for finding the target personnel at the work site in the prior art cannot cover the entire work area and has a poor positioning effect can be solved by adjusting the kernel parameters.

[0097] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0098] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the control method of the terminal device.

[0099] Specifically, the control method of the terminal device includes:

[0100] Step S201, receiving real-time video data sent by the video acquisition device, wherein the real-time video data is video data including images of the terminal device and the user;

[0101] Step S202, receiving high-precision positioning data sent by the Beidou positioning receiver, the high-precision positioning data including the device positioning data of the terminal device and the user positioning data of the user;

[0102] Step S203, integrating the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generating a control instruction for the terminal device at least based on the comprehensive information data;

[0103] Step S204: sending the control instruction to the terminal device so that the terminal device performs the target operation according to the control instruction.

[0104] Optionally, the real-time video data and the high-precision positioning data are integrated to obtain comprehensive information data, including: pre-processing the real-time video data and the high-precision positioning data respectively to obtain processed video data and processed positioning data; and synchronously integrating the processed video data and the processed positioning data using a data fusion algorithm to obtain the comprehensive information data.

[0105] Optionally, the real-time video data and the high-precision positioning data are preprocessed respectively to obtain processed video data and processed positioning data, including: decoding the real-time video data to obtain decoded video data, wherein the decoded video data is a digital signal recognizable by the central processing unit; extracting key video information from the decoded video data using an image recognition and processing algorithm to obtain the processed video data, wherein the key video information includes at least one of the following: the user's position, the position, speed and movement direction of the target object, wherein the image recognition and processing algorithm includes at least a target detection algorithm, a target tracking algorithm and a target recognition algorithm, and the image recognition and processing algorithm is obtained based on deep learning technology or machine learning technology.

[0106] Optionally, the real-time video data and the high-precision positioning data are preprocessed respectively to obtain processed video data and processed positioning data, including: parsing the high-precision positioning data, extracting the longitude data and latitude data of the terminal device and the longitude data and latitude data of the user, and obtaining the processed positioning data, the accuracy of the processed positioning data being at the centimeter level.

[0107] Optionally, the control instructions of the terminal device are generated at least based on the comprehensive information data, including: generating a driving path of the terminal device based on the comprehensive information data, where there are no users and target objects on the driving path of the terminal device; determining an operation strategy of the terminal device within a target time period based on the driving path of the terminal device, where the operation strategy at least includes an operation direction and an operation speed of the terminal device at each moment within the target time period; receiving a user instruction, where the user instruction is generated based on information input into a user interface by the user; determining a control instruction of the terminal device based on the user instruction, the driving path of the terminal device and the operation strategy of the terminal device within the target time period.

[0108] Optionally, the terminal device is a vehicle, the comprehensive information data includes status data of the target object, and the control instruction of the terminal device is generated according to the comprehensive information data, including: when the target object is an indicator light, and the status data of the target object is color data of the indicator light, the control instruction of the terminal device is determined according to the color data of the indicator light, wherein when the color of the indicator light is red, the control instruction of the terminal device is a stop instruction; when the target object is an obstacle, and the status data of the target object is position data of the obstacle, the control instruction of the terminal device is determined according to the position data of the obstacle to avoid a collision between the terminal device and the obstacle, wherein the control instruction of the terminal device includes the running direction and speed of the terminal device.

[0109] Optionally, after receiving the high-precision positioning data of the terminal device sent by the Beidou positioning receiver, the method further includes: determining whether there is a collision risk between the terminal device and the user based on the real-time video data and the high-precision positioning data; and automatically adjusting the operating state of the terminal device and / or sending an alarm message when there is a collision risk between the terminal device and the user. An embodiment of the present invention provides a processor, the processor is used to run a program, wherein the control method of the terminal device is executed when the program is running.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0111] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0120] 1) The control method of the above-mentioned terminal device of the present application first receives the real-time video data sent by the video acquisition device, and the real-time video data is the video data including the images of the terminal device and the user; then receives the high-precision positioning data sent by the Beidou positioning receiver, and the high-precision positioning data includes the device positioning data of the terminal device and the user positioning data of the user; then integrates the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generates the control instructions of the terminal device at least according to the comprehensive information data; finally sends the control instructions to the terminal device, so that the terminal device performs the target operation according to the control instructions. The method can obtain the real-time image and precise positioning of the device, realize the linkage of video and positioning technology, so as to accurately locate the position and real-time dynamics of the target personnel, and can accurately integrate the video data and positioning data and output the control instructions of the terminal device, realize the linkage control of the terminal positioning and the video device in three-dimensional space, solve the problem that the detection method of finding the target personnel at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, and improve the flexibility and accuracy of video monitoring, thereby comprehensively improving the safety management and control of the operation site and ensuring the safety of personnel during the operation.

[0121] 2) The control system of the above-mentioned terminal equipment of the present application includes: a video acquisition device for capturing real-time video data; a Beidou positioning receiver for receiving Beidou satellite signals and outputting high-precision positioning data; a central processing unit for receiving and processing video data from the video acquisition device and positioning data from the Beidou positioning receiver, and generating control instructions based on these data to control the remote terminal equipment. The system can obtain real-time images and precise positioning of the equipment, realize the linkage of video and positioning technology, thereby accurately locating the position and real-time dynamics of the target personnel, and can accurately integrate the video data and positioning data and output the control instructions of the terminal equipment, realize the linkage control of terminal positioning and video equipment in three-dimensional space, solve the problem that the detection method for finding the target personnel at the operation site in the prior art cannot cover the entire operation area and the positioning effect is poor, improve the flexibility and accuracy of video monitoring, and improve the response speed and reliability of the overall system, thereby comprehensively improving the safety management and control of the operation site and ensuring the safety of personnel during the operation.

[0122] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for controlling a terminal device, characterized in that: A central processing unit applied to a terminal linkage system, wherein the terminal linkage system further comprises a video acquisition device, a Beidou positioning receiver and a terminal device, wherein the central processing unit is respectively connected to the video acquisition device, the Beidou positioning receiver and the terminal device for communication, and the method comprises: Receiving real-time video data sent by the video acquisition device, wherein the real-time video data is video data including images of the terminal device and the user; Receiving high-precision positioning data sent by the Beidou positioning receiver, the high-precision positioning data including device positioning data of the terminal device and user positioning data of the user; Integrate the real-time video data and the high-precision positioning data to obtain comprehensive information data, and generate a control instruction for the terminal device at least based on the comprehensive information data; The control instruction is sent to the terminal device, so that the terminal device performs the target operation according to the control instruction.

2. The method according to claim 1, characterized in that The real-time video data and the high-precision positioning data are integrated to obtain comprehensive information data, including: Preprocessing the real-time video data and the high-precision positioning data respectively to obtain processed video data and processed positioning data; The processed video data and the processed positioning data are synchronously integrated using a data fusion algorithm to obtain the comprehensive information data.

3. The method according to claim 2, characterized in that Preprocessing the real-time video data and the high-precision positioning data respectively to obtain processed video data and processed positioning data includes: Decoding the real-time video data to obtain decoded video data, wherein the decoded video data is a digital signal recognizable by the central processing unit; An image recognition and processing algorithm is used to extract video key information from the decoded video data to obtain the processed video data, wherein the video key information includes at least one of the following: the user's position, the target object's position, speed and movement direction, and the image recognition and processing algorithm includes at least a target detection algorithm, a target tracking algorithm and a target recognition algorithm, and the image recognition and processing algorithm is obtained based on deep learning technology or machine learning technology.

4. The method according to claim 2, characterized in that: Preprocessing the real-time video data and the high-precision positioning data respectively to obtain processed video data and processed positioning data includes: The high-precision positioning data is parsed to extract the longitude data and latitude data of the terminal device and the longitude data and latitude data of the user to obtain the processed positioning data, the accuracy of the processed positioning data is at the centimeter level.

5. The method according to claim 1, characterized in that Generating a control instruction of the terminal device at least according to the comprehensive information data includes: generating a driving path of the terminal device according to the comprehensive information data, wherein no user or target object exists on the driving path of the terminal device; Determine, according to the driving path of the terminal device, an operation strategy of the terminal device within a target time period, wherein the operation strategy at least includes an operation direction and an operation speed of the terminal device at each time within the target time period; receiving a user instruction, wherein the user instruction is generated according to information input into a user interface by the user; The control instruction of the terminal device is determined according to the user instruction, the driving path of the terminal device and the operation strategy of the terminal device in the target time period.

6. The method according to claim 1, characterized in that The terminal device is a vehicle, the comprehensive information data includes state data of the target object, and the control instruction of the terminal device is generated according to the comprehensive information data, including: When the target object is an indicator light and the state data of the target object is color data of the indicator light, the control instruction of the terminal device is determined according to the color data of the indicator light, wherein when the color of the indicator light is red, the control instruction of the terminal device is a stop instruction; When the target object is an obstacle and the status data of the target object is the position data of the obstacle, the control instruction of the terminal device is determined according to the position data of the obstacle to avoid a collision between the terminal device and the obstacle, wherein the control instruction of the terminal device includes the running direction and speed of the terminal device.

7. The method according to any one of claims 1 to 6, characterized in that: After receiving the high-precision positioning data of the terminal device sent by the Beidou positioning receiver, the method further includes: Determining whether there is a collision risk between the terminal device and the user according to the real-time video data and the high-precision positioning data; When there is the risk of collision between the terminal device and the user, the operating state of the terminal device is automatically adjusted and / or an alarm message is sent.

8. A terminal linkage system, characterized in that: include: A central processing unit, configured to execute the control method of the terminal device according to any one of claims 1 to 7; A video acquisition device, used to acquire video data including images of the terminal device and the user, the video acquisition device being communicatively connected to the central processing unit; A Beidou positioning receiver, used for collecting device positioning data of the terminal device and user positioning data of the user, the Beidou positioning receiver being communicatively connected with the central processing unit; The terminal device is communicatively connected with the central processing unit.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the terminal device control method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a control method for executing the terminal device described in any one of claims 1 to 7.

Images