Video-based job vehicle measurement method and system

Abstract

This invention discloses a video-based method and system for measuring work vehicles. The method includes, in response to receiving a monitoring request for a work area to be measured from a user terminal, invoking a virtual scene map mapped to the work area to be measured; when the work area to be measured enters the measurement state, acquiring a real-time video stream of the work area; and automatically mapping the work vehicles in the real-time video stream to the virtual scene map to measure the real-time work data of the work vehicles. By establishing a virtual scene map of the work area and updating the position of the work vehicles in the virtual scene map in real time based on the video stream, the measurement of work vehicles is realized in the virtual scene map. This improves the accuracy of work vehicle data measurement during operation, reduces the cost of work vehicle measurement data, and thus solves the problems of low measurement accuracy, high measurement difficulty, and high measurement cost in related technologies.

Description

Technical Field

[0001] This invention relates to the field of vehicle measurement technology, and specifically to a video-based method and system for measuring operational vehicles. Background Technology

[0002] Specialized vehicles, such as cranes and forklifts, typically require data to be recorded during operation to determine whether they are operating within safe limits. For example, when using cranes or forklifts to assemble or dismantle utility poles or lift and install large power facilities, the surrounding area often contains energized electrical equipment such as power lines and transformers. Therefore, there is a significant risk of electric shock and fire to vehicles and personnel from contact with these electrical facilities during lifting or rotation. Thus, data measured by the vehicles is crucial for determining whether they are operating within safe limits.

[0003] In related technologies, the measurement of operational data of work vehicles, such as special vehicles, is usually obtained by deploying a large range of high-density sensors. This method of measuring data is extremely costly, and because the accuracy of data measurement depends on the performance of the sensors, the measurement accuracy of work vehicles is low. Summary of the Invention

[0004] The main objective of this invention is to provide a video-based method and system for measuring work vehicles.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a video-based method for measuring work vehicles is provided, comprising: responding to receiving a measurement request from a user terminal for a work area to be measured, invoking a virtual scene map mapped to the work area to be measured; when the work area to be measured enters a measurement state, acquiring a real-time video stream of the work area to be measured; and automatically mapping the work vehicles in the real-time video stream to the virtual scene map after automatically calibrating them, so as to measure the real-time work data of the work vehicles.

[0006] Optionally, the method further includes: when the work vehicle is mapped to the virtual scene map, determining the target work area of ​​the work vehicle based on the safe work range set in the virtual scene map; and planning work parameters for the work vehicle based on the target work area.

[0007] Optionally, it further includes: when the work vehicle is performing operations based on the work parameters, updating the pose data of the work vehicle mapped in the virtual scene map based on the acquired real-time video stream; and determining in real time whether the pose data is consistent with the safe operating range.

[0008] Optionally, it also includes: if they do not match, remotely controlling the work vehicle to stop working and re-planning the work parameters for the work vehicle.

[0009] Optionally, it further includes: generating a virtual scene map mapping the area to be measured based on the data of the charged body in the area to be measured and the positioning data of the non-charged body area; and defining the safe operation range in the virtual scene map based on the sub-safe operation range corresponding to each different charged body in the area to be measured.

[0010] Optionally, it further includes: if the virtual scene map includes multiple work vehicles, determining the target work area corresponding to each work vehicle based on a pre-established work collaboration model; and planning the work parameters for each work vehicle based on each target work area.

[0011] Optionally, the operation collaboration model determines the constraint condition that the operation vehicle does not fall into the non-safe operation area based on the operation vehicle pose information and the safe operation range in the operation area; and determines the loss function based on the operation vehicle pose information.

[0012] Optionally, automatically calibrating the work vehicles in the real-time video stream includes: automatically calibrating the positioning position of the work vehicles in the real-time video stream based on preset distance calculation rules, wherein the positioning position is used to map the work vehicles to preset positions in the virtual scene map; and / or, automatically calibrating the type of work vehicles in the real-time video stream based on image recognition algorithms, wherein the different types of work vehicles correspond to different work parameters.

[0013] Optionally, the automatic calibration of the work vehicles in the real-time video stream includes: automatically calibrating the positioning position of the work vehicles in the real-time video stream based on a preset distance calculation rule, wherein the positioning position is used to map the work vehicles to a preset position in the virtual scene map; and / or, automatically calibrating the type of the work vehicles in the real-time video stream based on an image recognition algorithm, wherein different types of work vehicles correspond to different work parameters.

[0014] According to a second aspect of the present invention, a video-based measurement system for work vehicles is provided. A calling unit is configured to, in response to receiving a monitoring request for a work area to be measured sent by a user terminal, call a virtual scene map mapped to the work area to be measured; a video stream acquisition unit is configured to acquire a real-time video stream of the work area to be measured when the work area to be measured enters a measurement state; and a monitoring unit is configured to, after automatically calibrating the work vehicles in the real-time video stream, automatically map the work vehicles to the virtual scene map to measure the real-time work data of the work vehicles.

[0015] According to a third aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores a computer program executable by said at least one processor, said computer program being executed by said at least one processor to cause said at least one processor to perform the video-based work vehicle measurement method according to any implementation of the first aspect.

[0016] The video-based method and system for measuring work vehicles of the present invention includes, in response to receiving a monitoring request for a work area to be measured sent by a user terminal, invoking a virtual scene map mapped to the work area to be measured; when the work area to be measured enters the measurement state, acquiring a real-time video stream of the work area to be measured; after automatically calibrating the work vehicles in the real-time video stream, automatically mapping the work vehicles to the virtual scene map to measure the real-time work data of the work vehicles. By establishing a virtual scene map of the work area and updating the position of the work vehicles in the virtual scene map in real time based on the video stream, the measurement of work vehicle data in the virtual scene map is realized. Based on the measured data, the monitoring of work vehicles can be realized, improving the accuracy of the measurement data of work vehicles during operation, reducing the cost required for measuring work vehicle data, and thus solving the problems of low measurement accuracy, high measurement difficulty, and high measurement cost in related technologies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a video-based measurement method for work vehicles according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of a video-based measurement system for work vehicles according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] According to embodiments of the present invention, a video-based method for measuring work vehicles is provided, such as... Figure 1 As shown, the method includes the following steps 101 to 103:

[0025] Step 101: In response to receiving the monitoring request for the work area to be measured sent by the user terminal, call the virtual scene map mapped to the work area to be measured.

[0026] In this embodiment, the virtual scene map mapped to the work area can be divided into multiple sub-virtual scene maps. Each sub-virtual scene map can be stored with a custom name as an index identifier, such as "National Highway 107", so that any sub-virtual scene map can be called through the index identifier.

[0027] Users can select one or more sub-virtual scene maps based on the area to be measured through the user interface to trigger a monitoring request. The execution entity can then invoke the sub-virtual scene map based on the request instruction name after receiving the request. After selecting a sub-virtual scene map in a customized manner, a start command can be sent to the corresponding work vehicle in the area to be measured, causing the work vehicle to enter the working state, i.e., the measurement state. This customized method allows for flexible control of the measurement of work vehicles within the work area.

[0028] As an optional implementation of this embodiment, generating a virtual scene map of the area to be measured based on the surveying data corresponding to the work area to be measured includes: generating a virtual scene map of the area to be measured based on the data of the charged bodies in the work area to be measured and the positioning data of the non-charged body areas; and delineating the safe work area in the virtual scene map based on the safe work area corresponding to each different charged body in the work area to be measured.

[0029] In this optional implementation, a virtual scene map can be generated based on surveying data, where the surveying data is determined by video captured by a camera device, i.e., video surveying. For example, camera devices can be installed at different points in the work area to acquire images of the work area. Based on the acquired images, a preset algorithm (e.g., monocular ranging) can be used to determine the positional parameters (including but not limited to distance and offset relative to the camera) of each object in the work area relative to the camera. Then, the position of each object in the work area can be determined by combining the camera's position within the work area. Based on the positions of each object, a virtual scene map is generated using a virtual scene map generation algorithm. The generated virtual scene map can be stored in a database and retrieved. The aforementioned surveying data includes data on charged bodies (including position and attitude data of the charged bodies, structural data representing the outline or size of the charged bodies), and also includes positioning data of non-charged body areas (including distances between various points within the area, or structural data of other objects within the area, such as obstacles).

[0030] Furthermore, to make the obtained virtual scene map more accurate, it is corrected using data pre-collected on the work area using an infrared distance meter after the virtual scene map is obtained. The collected data includes the location of live parts within the work area, and other spatial range data besides the location of live parts, etc.

[0031] The safe operating range varies for different electrical equipment (energized parts). For example, the safe operating range for electrical equipment such as transformers, busbars, and transmission and distribution lines can be determined based on the equipment structure, voltage level, energized parts, and the safe distance specified in the electrical safety regulations (the safe distance is generally determined by the voltage level, with 0.7m for 10 kV and below, 1m for 33 kV, 1.3m for 110 kV, and 3m for 220 kV). Outside the safe operating range, there is usually a space where people or objects are prohibited.

[0032] The system allows for the pre-determination of safe distances for different types of live conductors. After obtaining a virtual scene map, it identifies the live conductors within the map and determines their safe distances based on their type. The system then automatically delineates a safe work area within the virtual scene map; areas outside this safe distance (e.g., beyond 1 meter) are designated as safe work zones. This delineation can be achieved by highlighting the areas within the safe distance (prohibited work zones) with a prominent color. After determining the safe distance, the monitoring range can be expanded via the interactive interface (e.g., expanding 1 meter to 2 meters, creating a circular area with a 2-meter radius). Once a work vehicle enters this monitoring range, subsequent alarm actions can be triggered, such as transmitting remote commands to the vehicle to provide a warning.

[0033] As an optional implementation of this embodiment, before calling the virtual scene map mapped to the region to be measured, the method further includes generating the virtual scene map mapped to the region to be measured based on a preset digital twin algorithm, wherein the virtual scene map includes a digital twin three-dimensional scene map.

[0034] In this optional implementation, digital twin technology can be used to map the area to be measured onto a three-dimensional map according to the actual parameters.

[0035] Step 102: Once the work area to be measured enters the measurement state, acquire the real-time video stream of the work area to be measured.

[0036] In this embodiment, the measurement state can be entered by triggering the component to start the measurement. After entering the measurement state, the camera devices at each point in the area to be monitored can be turned on to obtain real-time video streams.

[0037] Step 103: After automatically calibrating the work vehicles in the real-time video stream, automatically map the work vehicles onto the virtual scene map to measure the real-time work data of the work vehicles.

[0038] In this embodiment, after acquiring the real-time video stream, the work vehicle can be calibrated in real-time based on the video stream to map it onto the virtual scene map. Furthermore, after mapping, the work vehicle can be monitored in real-time based on the virtual scene map. The monitoring method includes monitoring whether the work vehicle enters an unsafe working area containing live electrical components.

[0039] This embodiment can obtain measurement data of the work vehicle during operation through the above method. Based on the measurement data, it can be monitored to ensure that it operates within the monitoring range.

[0040] As an optional implementation of this embodiment, when the work vehicle is mapped to the virtual scene map, the target work area of ​​the work vehicle is determined based on the safe work range set in the virtual scene map; and work parameters are planned for the work vehicle based on the target work area.

[0041] In this optional implementation, after the virtual scene map is invoked and displayed on the interactive interface, the prohibited work area is automatically determined based on the live parts in the virtual scene map, and this area is highlighted in the virtual scene map, such as by marking it in red. After the prohibited work area is determined, a preset spatial range can be expanded on the basis of the prohibited work area to serve as the monitoring range. Once the work vehicle enters the monitoring range, subsequent alarm actions can be executed, such as transmitting remote commands to the work vehicle to achieve a warning effect.

[0042] The area outside the monitoring range can be considered as the safe operating range of this implementation. When the area to be monitored contains multiple power devices, the final safe operating range is determined based on the safe operating range of the multiple power devices. The method of determination can be implemented by logical operations between sets, which is not limited here.

[0043] Furthermore, different work vehicles possess different pose data. Therefore, when a work vehicle is mapped onto the virtual scene map, the final safe working target area for that vehicle can be determined based on the safe working range and the vehicle's position data. For example, within the safe working range, the final workable area for the same work vehicle at different points, such as point A and point B, will be different. Similarly, different work vehicles at the same point will also have different final workable areas. It should be understood that the calculation method for determining the final safe working target area based on the safe working range and the vehicle's position data can employ set logic calculations, or other methods, which are not limited here. It is understood that the type of work vehicle is determined based on the real-time video stream, and based on that type, a corresponding 3D model is retrieved from a pre-established model library and mapped to a preset position in the virtual scene map.

[0044] Furthermore, different work vehicles have different types of operating parameters. For example, for crane trucks, the corresponding operating parameters may include the length of the boom extension, the angle of the boom extension, the height of the boom, and the angle of rotation. When the work vehicle is in operation, these different parameters will lead to changes in the range of working space.

[0045] To ensure that the crane operates within a safe operating range, its operating parameters need to be planned based on the vehicle's specific parameters and the defined target operating area. Each parameter can be a threshold value allowed for safe operation, representing the permissible threshold for each operating part of the vehicle; it can be a maximum or minimum value. For example, if the target operating area is A, crane operations can utilize parameters such as boom extension length, boom extension angle, boom height, and rotation angle to complete the task. These parameters can be planned, such as a maximum boom extension length of 2m, a maximum boom extension angle of 100 degrees, a boom height not exceeding 5m, and a rotation angle not less than 30 degrees in target area A. The crane then operates only when these thresholds are met.

[0046] Therefore, this embodiment can plan operating parameters for different operating vehicles based on the target operating area, so that the operating vehicles can perform operations based on the operating parameters. Furthermore, different operating vehicles can have different operating parameters. Taking special operating vehicles as an example, operating parameters can include, but are not limited to, travel route, allowable angle of the boom, height, and rotation angle.

[0047] This optional implementation method enables the work vehicles to operate safely within a safe range by automatically planning their work data.

[0048] As an optional implementation of this embodiment, automatically calibrating the work vehicles in the real-time video stream includes: automatically calibrating the positioning position of the work vehicles in the real-time video stream based on a preset distance calculation rule, wherein the positioning position is used to map the work vehicles to a preset position in the virtual scene map; and / or, automatically calibrating the type of work vehicles in the real-time video stream based on an image recognition algorithm, wherein the different types of work vehicles correspond to different work parameters.

[0049] In this optional implementation, after acquiring the video stream, the location and type of the work vehicle are determined. Once the type of work vehicle is determined, the corresponding 3D model can be called from a pre-established model library. The structural parameters of this model are the same as the actual parameters of the work vehicle.

[0050] As an optional implementation of this embodiment, if the virtual scene diagram includes multiple work vehicles, the target work areas of each work vehicle are determined based on the pre-established work collaboration model; and work parameters are planned for each work vehicle based on each target work area.

[0051] In this optional implementation, multiple work vehicles can coexist in the virtual scene map. When a work vehicle is working according to the planned work data, the position and pose data of the surrounding work vehicles may affect its work range. For example, if the adjacent work vehicles are all cranes, the position of its boom during the lifting process may be affected by the parking position of other cranes or the position data of the boom of other cranes during the lifting process.

[0052] As an optional implementation method in this embodiment, the operation collaboration model determines the constraint condition that the operation vehicle does not fall into the non-safe operation area based on the operation vehicle pose information and the safe operation range in the operation area; and determines the loss function based on the operation vehicle pose information.

[0053] In this optional implementation, the pose information of the vehicle to be planned can be input into the operation coordination model, which then outputs the planned operation parameters for the vehicle. This operation coordination model, based on constraint-based operation data, ensures that any vehicle operates within a safe range, and, based on a loss function, guarantees that collisions do not occur between vehicles, thereby maximizing the efficiency of completing the task.

[0054] As an optional implementation of this embodiment, when the work vehicle is performing operations based on the work parameters, the pose data of the mapped work vehicle is updated in the virtual scene map based on the real-time acquired video stream; and it is determined in real time whether the pose data is consistent with the safe working range.

[0055] In this optional implementation, although the planned work data ensures that the work vehicle operates within a safe working range, the precision of the mechanical actions of the work vehicle based on the work parameters during operation may be insufficient. For example, if the crane boom momentarily exceeds the maximum lifting height during lifting, it may enter an unsafe area. Therefore, this optional implementation uses real-time monitoring to provide timely warnings for work vehicles that may enter unsafe areas. Due to the complexity of the work site conditions and the variety of work vehicles, it is impossible to adjust the work vehicle's work parameters in real time (within a very short time), making this method highly complex. Therefore, the real-time monitoring in this embodiment aims to trigger automatic monitoring of the work vehicle at preset intervals, thereby achieving continuous monitoring throughout the entire operation. It should be understood that the preset interval value may vary depending on the type of work vehicle and the work task. Since the work vehicle operates according to the aforementioned planned parameters at the start of operation, and these work parameters represent the allowable thresholds for each working part of the work vehicle, monitoring is performed at preset intervals to ensure that the work vehicle operates within these threshold ranges. This method enables real-time monitoring of work vehicles while reducing the complexity of monitoring methods.

[0056] During the operation of the work vehicle, real-time video streams (video streams at the beginning of the next monitoring cycle) are acquired, and the position and pose data of the work vehicle are calibrated based on the real-time video. The position and pose data, as well as the aforementioned determined monitoring range, are judged in real time to determine whether the position and pose data falls within the monitoring range. If it falls within the range, it is inconsistent with the safe operation range. If it falls within the range, the planning parameters for the next cycle are re-planned for the work vehicle. If it does not fall within the range, the planning parameters for the current cycle continue to be used.

[0057] As an optional implementation of this embodiment, if the parameters do not match, the remotely controlled work vehicle will stop operating and the work parameters will be re-planned for the work vehicle.

[0058] In this optional implementation, if the position data of the work vehicle does not match the safe working range data, a remote notification command is sent to the work vehicle (or the terminal of the work vehicle) to notify the work vehicle to stop working and to plan the planning parameters for the next cycle for the work vehicle.

[0059] This embodiment establishes a virtual scene map of the work area and updates the position of the work vehicle in the virtual scene map in real time based on the video stream, thereby realizing the monitoring of the work vehicle in the virtual scene map, improving the monitoring accuracy and realizing automated monitoring.

[0060] The present invention also provides another embodiment of the operation collaboration model, including: establishing constraints on the operation vehicle not falling into the non-safe operation area based on the position and pose information of the operation vehicle and the safe operation range information in the operation area; it should be understood that the functional equations constituting the constraints may be different for different types of operation vehicles (electric equipment) and different operation scenarios, which are not limited here, and the purpose of achieving the constraints through functional equations all fall within the scope of protection disclosed in this embodiment.

[0061] Based on the pose information of the work vehicles, a first equation J1 is established for each work vehicle to avoid collisions. This first equation J1 reflects the equation for avoiding collisions between any two work vehicles. Due to the special nature of work vehicles, such as special vehicles differing from ordinary vehicles, the distances between the work vehicles include not only the distance between them when moving in a two-dimensional plane (i.e., on the ground), but also the distance between the work parts of the work vehicles when moving in three-dimensional space (i.e., space above the two-dimensional plane). This distance can be calculated by projecting each point of the work part in three-dimensional space onto a two-dimensional plane, and then determining the calculation equation for avoiding collisions through the set of points. This equation indicates that no collisions occur during vehicle movement and during spatial operations. This is merely an example, and the equations can be modified without departing from the scope of this embodiment.

[0062] It also includes establishing a second equation J2 based on the vehicle's pose information regarding the energy consumption required for the vehicle to complete its task. The energy consumption in this second equation is determined by a set of planned task data values. Since the smaller the action required by each working part of the vehicle, the lower the energy consumption, for example, the energy consumption required for a 15° rotation is less than that required for a 30° rotation. When planning task parameters for each vehicle, the planned parameters can include multiple sets of planning data, such as {length 2m, maximum unfolded angle 100°, height 5m, rotation angle 30°} or {length 1m, maximum unfolded angle 90°, height 4m, rotation angle 40°}. Different task parameters correspond to different energy consumption values. Based on each set of task parameters and the unit energy consumption value of each part of the vehicle, the energy consumption value corresponding to multiple sets of planning data can be determined. The group corresponding to the minimum energy consumption value is used as the planning data for the working vehicle, and this planning data is used as the maximum energy consumption in the second equation. That is, during the operation, the energy consumption value of the working vehicle is determined by the change of pose information within the above monitoring period. The second equation ensures that the actual energy consumption value is less than the maximum energy consumption in the second equation.

[0063] The first and second equations can be used as loss functions. It should be understood that the equations for the above loss functions can be set according to the different types of operating vehicles and the actual operating scenarios, and are not limited here. However, achieving the purpose of this loss function through functional equations falls within the scope of protection disclosed in this embodiment.

[0064] The embodiments of the present invention also include the steps of implementing the operation coordination method based on the above-mentioned operation coordination model, including obtaining the position and pose information of any operation vehicle when it is in operation, inputting the position and pose information into the above-mentioned model, and outputting the planned operation parameters of the operation vehicle.

[0065] By planning the operation parameters through the operation collaboration model in this embodiment, the operation vehicles can be guaranteed to operate safely while also achieving the maximum operation efficiency.

[0066] This embodiment implements a collaborative operation model that automates the planning of operational parameters for work vehicles, avoiding the inability to completely prevent safety accidents due to manual estimation of operational data. Furthermore, this model supports operational parameter planning in scenarios with multiple work vehicles, improving operational efficiency and ensuring the safety of each vehicle.

[0067] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0068] According to an embodiment of the present invention, a video-based measurement system for work vehicles is also provided. The system includes: a calling unit 201, configured to call a virtual scene map mapped to the work area to be measured in response to receiving a measurement request for the work area to be measured sent by a user terminal; a video stream acquisition unit 202, configured to acquire a real-time video stream of the work area to be measured after the work area to be measured enters the measurement state; and a measurement unit 203, configured to automatically map the work vehicle to the virtual scene map after automatically calibrating the work vehicle in the real-time video stream, so as to measure the real-time work data of the work vehicle.

[0069] As an optional implementation of this embodiment, the system further includes a determining unit, configured to determine the target operating area of ​​the operating vehicle based on the safe operating range set in the virtual scene map when the operating vehicle is mapped to the virtual scene map; and a planning unit, configured to plan operating parameters for the operating vehicle based on the target operating area.

[0070] As an optional implementation of this embodiment, the system further includes an update unit, configured to update the pose data of the mapped work vehicle in the virtual scene graph based on the real-time acquired video stream when the work vehicle is performing work based on the work parameters; and to determine in real time whether the pose data is consistent with the safe working range.

[0071] As an optional implementation of this embodiment, if the parameters do not match, the remotely controlled work vehicle will stop operating and the work parameters will be re-planned for the work vehicle.

[0072] As an optional implementation of this embodiment, it further includes: a generation unit configured to generate a virtual scene map mapping the area to be measured based on the position data of the charged body in the work area to be measured, the structural data of the charged body, and the positioning data of the non-charged body area; and a delineation unit configured to delineate the safe work area in the virtual scene map based on the sub-safe work areas corresponding to each different charged body in the work area to be measured.

[0073] As an optional implementation of this embodiment, if the virtual scene diagram includes multiple work vehicles, the target work areas of each work vehicle are determined based on the pre-established work collaboration model; and work parameters are planned for each work vehicle based on each target work area.

[0074] As an optional implementation method in this embodiment, the operation collaboration model determines the constraint condition that the operation vehicle does not fall into the non-safe operation area based on the operation vehicle pose information and the safe operation range in the operation area; and determines the loss function based on the operation vehicle pose information.

[0075] As an optional implementation of this embodiment, automatically calibrating the work vehicles in the real-time video stream includes: automatically calibrating the positioning position of the work vehicles in the real-time video stream based on a preset distance calculation rule, wherein the positioning position is used to map the work vehicles to a preset position in the virtual scene map; and / or, automatically calibrating the type of work vehicles in the real-time video stream based on an image recognition algorithm, wherein the different types of work vehicles correspond to different work parameters.

[0076] This invention provides an electronic device, such as... Figure 3 As shown, the electronic device includes one or more processors 31 and a memory 32. Figure 3 Take a processor 31 as an example.

[0077] The controller may also include an input device 33 and an output device 34.

[0078] The processor 31, memory 32, input device 33, and output device 34 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0079] Processor 31 can be a Central Processing Unit (CPU). Processor 31 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0080] The memory 32, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the control method in the embodiments of this disclosure. The processor 31 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 32, thereby implementing the method of the above-described method embodiments.

[0081] The memory 32 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the processing device operated by the server. Furthermore, the memory 32 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 32 may optionally include memory remotely located relative to the processor 31, and these remote memories can be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] Input device 33 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the server's processing device. Output device 34 may include display devices such as a display screen.

[0083] One or more modules are stored in memory 32, and when executed by one or more processors 31, they perform actions such as... Figure 1 The method shown.

[0084] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the motor control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0085] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A video-based work vehicle measurement method, characterized by, include: Based on the data of charged bodies in the area to be measured and the positioning data of non-charged body areas, a virtual scene map mapping the area to be measured is generated; Based on the sub-safety work ranges corresponding to different charged bodies in the work area to be measured, the safe work range is delineated in the virtual scene diagram; In response to receiving a measurement request from the user terminal for the area to be measured, the virtual scene map mapped to the area to be measured is invoked; Once the area to be measured enters the measurement state, a real-time video stream of the area to be measured is acquired. Automatically calibrating the operating vehicles in the real-time video stream includes: Based on preset distance calculation rules, the positioning position of the work vehicle in the real-time video stream is automatically determined, wherein the positioning position is used to map the work vehicle to a preset position in the virtual scene map; And / or, based on an image recognition algorithm, the type of the work vehicle in the real-time video stream is automatically identified, wherein different types of work vehicles correspond to different work parameters; After automatically calibrating the work vehicles in the real-time video stream, the work vehicles are automatically mapped to the virtual scene map. When the work vehicles are mapped to the virtual scene map, the target work area of ​​the work vehicles is determined based on the safe work range set in the virtual scene map. Based on the target work area, work parameters are planned for the work vehicle; When the work vehicle performs operations based on the work parameters, the pose data of the work vehicle mapped in the virtual scene map is updated based on the acquired real-time video stream. The system determines in real time whether the pose data matches the safe operating range in order to measure the real-time operating data of the operating vehicle.

2. The video-based job vehicle measurement method of claim 1, wherein, Also includes: If they do not match, the operation vehicle will be stopped remotely, and the operation parameters will be re-planned for the operation vehicle.

3. The video-based job vehicle measurement method of claim 1, wherein, Also includes: If the virtual scene map includes multiple work vehicles, the target work area corresponding to each work vehicle is determined based on the pre-established work collaboration model; Based on each of the target work areas, the work parameters are planned for each of the work vehicles.

4. The video-based measurement method for work vehicles according to claim 3, characterized in that, The collaborative operation model determines the constraints that prevent the operation vehicle from falling into the unsafe operation area based on the vehicle's pose information and the safe operation range in the operation area; and determines the loss function based on the vehicle's pose information.

5. A work vehicle measurement system characterized by, For performing the video-based measurement method for work vehicles as described in any one of claims 1-4, the work vehicle measurement system comprises: The calling unit is configured to call the virtual scene map mapped to the area to be measured in response to receiving a monitoring request for the work area to be measured sent by the user terminal; The video stream acquisition unit is configured to acquire the real-time video stream of the work area to be measured after the work area to be measured enters the measurement state. The measurement unit is configured to automatically map the work vehicle to the virtual scene map after automatically calibrating the work vehicle in the real-time video stream, so as to monitor the real-time work data of the work vehicle.

6. An electronic device, comprising: include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the video-based work vehicle measurement method according to any one of claims 1-4.

Images