Target following method and device for dome camera

Abstract

The invention relates to a target following method of a dome camera, the dome camera comprises a monitoring system and a control system, a target is a portal crane grab bucket, and the method comprises the following steps: the monitoring system identifies a position center point of the portal crane grab bucket through a target detection model obtained through convolutional neural network training, acquiring the current movement direction of the portal crane grab bucket and the current visual angle of the spherical camera, judging whether the central point of the portal crane grab bucket is in a correct range frame or not, and sending a control instruction to the control system according to a judgment result; the control system receives the rotation speed and the rotation time through a communication protocol, and controls the dome camera to rotate and suspend motion. The invention also provides a corresponding target following device of the dome camera. According to the invention, support for cooperation with different types of PLC systems can be provided in combination with the motion posture of the target, and the dome camera target following method which is timely in tracking, small in calculation amount and high in precision is provided.

Description

Technical Field

[0001] The present application relates to a technical field related to intelligent monitoring, and in particular to a target tracking method and device for a spherical camera. Background Art

[0002] Gantry cranes are often remotely controlled to load and unload large or heavy goods. Due to their large lifting capacity and complex structure, accidents may cause serious personnel and economic losses. In actual operations, monitoring equipment such as dome cameras are often used to monitor the operation of gantry cranes. However, when workers remotely operate gantry cranes, they are often unable to adjust the viewing angle of the dome camera at the same time. When encountering emergencies, they may not be discovered in time due to poor viewing angles, posing a safety hazard. An analysis of existing dome camera target tracking methods found that the existing technology may have the following disadvantages:

[0003] On the one hand, algorithms like CamShift track the target by calculating the probability distribution of target pixels. When the target and background features are not prominent, the accuracy of target recognition and tracking is not high.

[0004] On the other hand, the current target tracking control system does not take into account the target's current motion posture, so there may be problems with untimely tracking or misjudgment of direction in control.

[0005] Therefore, in this context, how the spherical camera can combine the target's own motion posture to provide a target following method with timely tracking, small amount of calculation and high precision is a technical problem to be solved. Summary of the invention

[0006] In view of the above problems of the prior art, the present application provides a target tracking method of a spherical camera, which can combine the motion posture of the target itself to provide a target tracking method with timely tracking, small amount of calculation and high precision.

[0007] To achieve the above-mentioned object, the first aspect of the present application provides a target tracking method of a spherical camera, wherein the target is a gantry crane grab bucket; the method comprises:

[0008] The spherical camera is used to obtain an image at a current viewing angle in real time, wherein the image includes the gantry crane grab bucket;

[0009] Identify the center point of the gantry crane grab bucket position in the image by using a target detection model;

[0010] Obtaining the current movement direction of the gantry crane grab bucket;

[0011] Determine whether the center point of the gantry crane grab bucket is within the correct range frame according to the current movement direction of the gantry crane grab bucket and the current viewing angle of the spherical camera;

[0012] According to the judgment result, the dome camera is controlled to rotate or pause.

[0013] From the above, the target detection model obtained through neural network training can improve the accuracy of target tracking; the control efficiency and accuracy can be improved by combining the current movement direction; by judging whether the center point of the gantry crane grab position is in the correct range box and obtaining the rotation speed and rotation time based on the results, the amount of calculation can be reduced and the tracking speed can be improved; communication with different systems through communication protocols can improve applicability.

[0014] As a possible implementation of the first aspect, the dome camera is mounted below the cab of the gantry crane and faces the grab bucket side of the gantry crane.

[0015] From the above, the dome camera can obtain image data including the gantry crane grab bucket.

[0016] As a possible implementation of the first aspect, the training method of the target detection model includes:

[0017] Collecting images of working scenes that may include the gantry crane grab to obtain an original data set;

[0018] Manually screening the original data set, removing images where the gantry crane grab bucket does not appear, and marking the range of the gantry crane grab bucket in the remaining images by selecting a box to obtain a marked data set;

[0019] Dividing the labeled data set into a training set and a test set according to a certain ratio;

[0020] The target detection model is trained using the training set, and the target detection model is verified using the test set.

[0021] From the above, the one-stage detection algorithm is used to convert the target detection problem into a regression problem, and the position of the prediction box and the object category contained in the prediction box are obtained from the regression result; therefore, the speed is faster, and the computational cost is lower due to the simplified processing flow; at the same time, compared with the probability distribution algorithm, the one-stage deep learning detection algorithm has higher accuracy and can meet the needs of real-time monitoring.

[0022] As a possible implementation of the first aspect, the target detection model may re-collect image data during use to continue training the target detection model.

[0023] From the above, by repeatedly collecting image data and training the target detection model, the judgment accuracy and speed can be gradually improved during actual use.

[0024] As a possible implementation of the first aspect, the center point (X 0 ,Y 0 ) is calculated based on the identification box selected after the maximum value suppression process, as follows:

[0025]

[0026] Among them, X 0 ,Y 0 are the X-axis and Y-axis coordinates of the center point of the gantry crane grab bucket, respectively. 1 ,Y 1 They are the X-axis and Y-axis coordinates of the upper left vertex of the gantry crane grab identification frame, 2 ,Y 2 They are respectively the X-axis and Y-axis coordinates of the lower right vertex of the gantry crane grab identification frame.

[0027] From the above, the identification frame selected after maximum suppression processing can improve the accuracy of monitoring and optimize the detection results; the gantry crane grab bucket identification frame is used to calculate the center point of the gantry crane grab bucket position for subsequent judgment steps, which has smaller calculation amount and faster operation, and is easy to use in real-time systems.

[0028] As a possible implementation of the first aspect, if the current movement direction is to the right relative to the current viewing angle, the correct range frame is a range frame with the center of the left half image of the current viewing angle as the center point; if the current movement direction is to the left relative to the current viewing angle, the correct range frame is a range frame with the center of the right half image of the current viewing angle as the center point; if the gantry crane grab bucket has not moved, the correct range frame is a range frame with the center of the current viewing angle image as the center point;

[0029] If the center point of the gantry crane grab bucket is within the correct range frame, the spherical camera does not need to be rotated; if the center point of the gantry crane grab bucket is not within the correct range frame, the spherical camera needs to be rotated, and the rotation time of the spherical camera is calculated by selecting a rotation speed of the spherical camera;

[0030] The rotation time of the dome camera is calculated according to the following formula:

[0031] X 0 -X′ 0 =V x t x ,Y 0 -Y′ 0 =V y t y

[0032] Among them, X′ 0,Y 0 ′ is the center point coordinate of the correct range frame, V x ,V y is the rotation speed in the x-axis and y-axis directions, t x ,t y is the rotation time in the x-axis and y-axis directions.

[0033] From the above, the judgment is made through the given correct range box, and the rotation speed is given. The required rotation time is calculated according to the coordinates, which reduces the amount of calculation and is faster.

[0034] As a possible implementation of the first aspect, the control system has multiple speed gears and can use different rotation speeds according to the position of the gantry crane grab in the current viewing angle.

[0035] As described above, setting multiple speed gears facilitates rapid positioning and tracking according to the movement speed and position of the gantry crane grab, thereby improving the tracking speed.

[0036] As a possible implementation method of the first aspect, after the rotation and pause movement is completed, the target detection and calculation are performed again to determine whether the set monitoring angle of view, that is, the correct range box, is reached; if not, the steps of calculating, sending control instructions, and controlling movement are continued; if it is reached, the movement of the spherical camera is stopped and wait for the next monitoring cycle.

[0037] From the above, a secondary judgment is made after the rotation movement, so that the target can be tracked continuously; when the target remains unchanged, the system can be put into standby mode to save system consumption.

[0038] A second aspect of the present application provides a target following device for a spherical camera, comprising:

[0039] A data collection module, used to obtain an image at a current viewing angle in real time through a dome camera, wherein the image includes a gantry crane grab bucket;

[0040] The data processing module identifies the center point of the gantry crane grab bucket in the image through the target detection model; obtains the current movement direction of the gantry crane grab bucket; and determines whether the center point of the gantry crane grab bucket is within the correct range frame according to the current movement direction of the gantry crane grab bucket and the current viewing angle of the dome camera;

[0041] The motion control module is used to control the spherical camera to rotate or pause according to the judgment result.

[0042] A third aspect of the present application provides a computing device, comprising: a processor, and a memory, on which program instructions are stored, and when the program instructions are executed by the processor, the processor executes the target tracking method for a spherical camera as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a first embodiment of the spherical camera target following method of the present application;

[0044] Figure 2 is a flow chart of a second embodiment of the spherical camera target following method of the present application;

[0045] Figure 3 is a schematic diagram of a spherical camera target following device provided in an embodiment of the present application;

[0046] Figure 4 It is a structural schematic diagram of a computing device provided in an embodiment of the present application.

[0047] It should be understood that the size and shape of each block diagram in the above structural diagram are for reference only and should not constitute an exclusive interpretation of the embodiment of the present invention. The relative position and inclusion relationship between the blocks presented in the structural diagram are only schematic representations of the structural association between the blocks, and do not limit the physical connection method of the embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution provided by the present application is further described below with reference to the accompanying drawings and examples. It should be understood that the system structure and business scenarios provided in the examples of the present application are mainly to illustrate the possible implementation methods of the technical solution of the present application and should not be interpreted as the only limitation on the technical solution of the present application. It is known to those of ordinary skill in the art that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0049] It should be understood that the target tracking solution for a spherical camera provided in the embodiments of the present application includes a target tracking method, device, computing device and readable storage medium for a spherical camera. Since the principles of solving the problems in these technical solutions are the same or similar, some repetitions may not be repeated in the following specific embodiments, but it should be considered that these specific embodiments have been referenced to each other and can be combined with each other.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of this application. In the event of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application. In order to accurately describe the technical content in this application and to accurately understand the present invention, the following explanations or definitions are given to the terms used in this specification before describing the specific embodiments:

[0051] 1) Programmable Logic Controller (PLC): A type of computer used in industrial automation and control systems; it can interact with sensors, actuators, and other devices through programmable control.

[0052] 2) One-Stage Object Detection Algorithm: This is a target detection method. Its core feature is that it can achieve target detection by extracting features only once. This type of algorithm does not require the region proposal stage and directly generates the category probability and position coordinate value of the object. The final detection result can be directly obtained after a single detection. Typical one-stage detection algorithms include YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) and RetinaNet.

[0053] The embodiment of the present application provides a target following solution for a spherical camera, which can obtain the image at the current viewing angle in real time through the spherical camera; identify the center point of the gantry crane grab position in the image through the target detection model; judge whether the center point of the gantry crane grab position is in the correct range frame in combination with the current movement direction of the gantry crane grab and the current viewing angle of the spherical camera; and control the spherical camera to rotate or pause according to the judgment result. This method can provide a target following method with timely tracking, small computational complexity and high precision in combination with the movement posture of the target itself. The embodiment of the present application can be applied to intelligent real-time monitoring in various fields such as construction, industry, security, and human-computer interaction. The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0054] The first embodiment of the present application provides a target tracking method for a spherical camera, wherein the target is a gantry crane grab. In some embodiments, the spherical camera may include a target detection model, a monitoring system, and a control system; the system structure is not limited here. Figure 1 , the implementation of each step of the method is specifically described, including steps S10-S50.

[0055] S10: acquiring an image at a current viewing angle in real time through the spherical camera, wherein the image includes the gantry crane grab bucket.

[0056] In some embodiments, the dome camera can track the grab bucket of the gantry crane, and can also track other moving objects, such as containers, operating personnel, other loading and unloading equipment, etc.

[0057] S20: Identify the center point of the gantry crane grab bucket position in the image through a target detection model.

[0058] In some embodiments, a target detection model is obtained by training a convolutional neural network to track the target; the target detection model can be trained, debugged, and verified using a one-stage detection algorithm; wherein the one-stage detection algorithm may include YOLO, SSD, RetinaNet, CenterNet, and other algorithms. In the process of training a convolutional neural network, the following steps may be included: collecting data, preprocessing data, enhancing data, annotating data, dividing data sets, designing network architecture, determining the number of layers and parameters, defining loss functions, selecting optimizers, setting learning rates, evaluating, fine-tuning, and the like.

[0059] In some embodiments, the target detection model may re-collect image data according to actual on-site conditions during use to continue training the target recognition model.

[0060] In some embodiments, the target detection model returns an identification frame of the gantry crane grab when performing gantry crane grab position identification, including the upper left vertex coordinates and the lower right vertex coordinates of the identification frame; wherein, the identification frame can be filtered through maximum suppression processing; the center point of the gantry crane grab position is calculated based on the upper left fixed point coordinates and the lower right vertex coordinates of the filtered identification frame.

[0061] S30: Acquire the current moving direction of the gantry crane grab bucket.

[0062] In some embodiments, the monitoring system obtains the current movement direction of the gantry crane grab by communicating with the PLC system; wherein, the monitoring system communicates with different types of PLC systems by configuring various types of communication protocols; wherein, the various types of communication protocols may include RTSP, ONVIF, SIP, TCP / IP, HTTP, UDP, etc., for transmitting and controlling video streams with the PLC system; wherein, the PLC system may use various types of integral, modular, stacked, etc. systems.

[0063] In some embodiments, the rotation mode of the spherical camera can be guided by obtaining the current movement direction of the gantry crane grab; in other embodiments, information such as the height of the gantry crane grab, the change in the boom mechanism, the rotation angle of the slewing mechanism, and the position of the grab from the rotation center point can also be obtained.

[0064] S40: judging whether the center point of the gantry crane grab bucket position is within a correct range frame according to the current movement direction of the gantry crane grab bucket and the current viewing angle of the spherical camera.

[0065] In some embodiments, the correct range box is a coordinate range given according to the current movement direction of the gantry crane grab under the current viewing angle of the monitoring system; for example, if the current movement direction is to the right relative to the current viewing angle, the correct range box is a range box with the center of the left half image of the current viewing angle as the center point; if the current movement direction is to the left relative to the current viewing angle, the correct range box is a range box with the center of the right half image of the current viewing angle as the center point; if the gantry crane grab has not moved, the correct range box is a range box with the center of the current viewing angle image as the center point.

[0066] In some embodiments, the monitoring system determines whether the center point of the gantry crane grab bucket is within the correct range frame by calculating the coordinates of the center point of the target grab bucket (X 0 ,Y 0 ) whether X exists 0 ∈(X′ 1 ,X′ 2 ),Y 0 ∈(Y′ 1 ,Y′ 2 ); where (X′ 1 ,X′ 2 ) represents the X-axis coordinate range of the correct scope box, (Y′ 1 ,Y′ 2 ) indicates the range of the Y-axis coordinate of the correct range frame. When the center point of the gantry crane grab bucket is within the correct range frame, the dome camera does not need to be rotated; when the center point of the gantry crane grab bucket is not within the correct range frame, the dome camera needs to be rotated.

[0067] S50: According to the judgment result, the dome camera is controlled to rotate or pause.

[0068] In some embodiments, based on the judgment results, the monitoring system sends control instructions to the control system. The control instructions may include rotation speed and rotation time; they may also include instruction parameters for related functions such as preset position points, motion trajectory recording, and safety alarms.

[0069] In some embodiments, when the dome camera is controlled using the rotation speed and the rotation time, the dome camera rotation time is calculated by automatically selecting a suitable dome camera rotation speed; for example, the dome camera rotation time can be calculated according to the following formula:

[0070] X 0 -X′ 0 =V x t x ,Y 0 -Y′ 0 =V y t y

[0071] Among them, X′0 ,Y 0 ′ is the coordinate of the center point of the correct range box, V x ,V y is the rotation speed in the x-axis and y-axis directions, t x ,t y is the rotation time in the x-axis and y-axis directions.

[0072] In some embodiments, the rotation speed can be selected according to the speed gear of the control system; when the current position of the gantry crane grab is far away from the target position and the movement speed is fast, a higher rotation speed of the spherical camera can be selected; when the current position of the gantry crane grab is close to the target position and the movement speed is slow, a lower rotation speed of the spherical camera can be selected.

[0073] In some embodiments, the control instructions acceptable to the dome camera include rotation speed and rotation time; and may also include instruction parameters for related functions such as preset position points, motion trajectory recording, and safety alarms.

[0074] In some embodiments, the control system has multiple speed gears, and different speeds can be used according to the position of the gantry crane grab in the current viewing angle.

[0075] In some embodiments, after the viewing angle of the spherical camera is adjusted, target detection and calculation are performed again to determine whether the set monitoring viewing angle is reached, that is, within the correct range frame of the gantry crane grab; if not, the steps of calculating, sending control instructions, and controlling movement are continued; if it is reached, the movement of the spherical camera is stopped and wait for the next monitoring cycle.

[0076] The second embodiment of the present application provides a method for tracking a gantry crane grab bucket using a dome camera. A dome camera installed near the gantry crane cab is used. The dome camera includes a monitoring system and a control system, and can obtain image data in different weather and working environments according to requirements. Figure 2 The method provided by the second embodiment includes the following steps S200-S250.

[0077] S200: Training the object detection model for gantry crane grab bucket through a one-stage deep learning model.

[0078] Firstly, the images that may contain the gantry crane grab bucket are collected by a spherical camera to obtain the original data set.

[0079] Then, we manually screened out images containing the gantry crane grab bucket target, and marked and selected its range in the selected images to obtain a labeled data set.

[0080] The data set is divided into a training set and a test set in a ratio of 4:1.

[0081] A one-stage detection model is designed and constructed, and the training set data is trained. According to the performance of the obtained model on the test set, the parameters and detection model are further adjusted until the accuracy of the test results reaches the standard. Finally, the target detection model of the gantry crane grab is obtained, which can return the range box of the gantry crane grab according to the input image.

[0082] During monitoring, the target detection model can re-collect image data and continue to train the target detection model.

[0083] S210: Identify the position of the gantry crane grab bucket according to the target detection model and calculate the center point of the position.

[0084] During the monitoring process of the dome camera, the monitoring system will transmit the images obtained through real-time monitoring to the PLC system through the communication protocol, and recognize the input images through the trained target detection model.

[0085] The returned target recognition results need to be processed by maximum value suppression to filter out the most suitable target frame, and calculate the center point coordinates of the door crane grab according to the following formula:

[0086]

[0087] Among them, X 0 ,Y 0 They are the X-axis and Y-axis coordinates of the center point of the gantry crane grab bucket. 1 ,Y 1 They are the X-axis and Y-axis coordinates of the upper left vertex of the gantry crane grab identification frame, 2 ,Y 2 They are respectively the X-axis and Y-axis coordinates of the lower right vertex of the gantry crane grab identification frame.

[0088] S220: The monitoring system obtains the current movement direction of the gantry crane grab bucket and the current viewing angle of the dome camera through the PLC system.

[0089] The monitoring system obtains the current movement direction of the gantry crane grab bucket by communicating with the PLC system, and obtains the current viewing angle of the dome camera through the RTSP protocol.

[0090] The current viewing angle of the monitoring system is transmitted to the ground workstation through the switch, played on the monitoring screen, and fed back to the relevant staff.

[0091] S230: The monitoring system determines whether the center point of the gantry crane grab bucket position is within the correct range frame according to the current movement direction.

[0092] The monitoring system will first select the correct range frame based on the current movement direction: if the current movement direction is to the right relative to the current viewing angle, the correct range frame is the range frame with the center of the left half image of the current viewing angle as the center point; if the current movement direction is to the left relative to the current viewing angle, the correct range frame is the range frame with the center of the right half image of the current viewing angle as the center point; if the gantry crane grab has not moved, the correct range frame is the range frame with the center of the current viewing angle image as the center point.

[0093] Then, the monitoring system will determine whether the center point of the gantry crane grab bucket position is within the correct range frame: if the center point of the gantry crane grab bucket position is within the correct range frame, the dome camera does not need to be rotated; if the center point of the gantry crane grab bucket position is not within the correct range frame, the dome camera needs to be rotated.

[0094] S240: According to the judgment result, select a suitable rotation speed and calculate the rotation time.

[0095] When the dome camera needs to rotate, the system will automatically select the appropriate rotation speed and calculate the rotation time of the dome camera.

[0096] The rotation time of the dome camera can be calculated according to the following formula:

[0097] X 0 -X′ 0 =V x t x ,Y 0 -Y′ 0 =V y t y

[0098] Among them, X′ 0 ,Y 0 ′ is the center point coordinate of the correct range frame, V x ,V y is the rotation speed in the x-axis and y-axis directions, t x ,t y is the rotation time in the x-axis and y-axis directions.

[0099] S250: The monitoring system sends a control command to the control system, and the control system controls the dome camera to adjust the viewing angle and check whether the gantry crane grab bucket reaches the correct range frame after moving.

[0100] The monitoring system sends the selected rotation speed and calculated rotation time to the control system. The control system receives movement instructions through the ONVIF protocol to control the rotation and pause of the dome camera.

[0101] After the rotation and pause movement are completed, steps S210-S230 are performed again to determine whether the set monitoring angle is reached, that is, the gantry crane grab is in the correct range frame. If not, continue with steps S240-S250; if it is reached, stop the movement of the dome camera and wait for the next monitoring cycle.

[0102] The third embodiment of the present application provides a target following device for a spherical camera, which can be used to implement the application compression method in the above embodiment, such as Figure 3 As shown, the target and device of the spherical camera include:

[0103] The data collection module is used to obtain the image at the current viewing angle in real time through the spherical camera, wherein the image includes the gantry crane grab; the data collection module can be used to implement step S10 in the first embodiment and its optional embodiments.

[0104] A data processing module is used to identify the center point of the gantry crane grab bucket in the image through a target detection model; obtain the current movement direction of the gantry crane grab bucket; and determine whether the center point of the gantry crane grab bucket is within a correct range frame based on the current movement direction of the gantry crane grab bucket and the current viewing angle of the spherical camera. The data collection module can be used to implement steps S20-S40 in the first embodiment and its optional embodiments.

[0105] The motion control module is used to control the dome camera to rotate or pause according to the judgment result. The data collection module can be used to implement step S50 in the first embodiment and its optional embodiments.

[0106] Figure 4 900 is a schematic structural diagram of a computing device 900 provided in an embodiment of the present application. The computing device can execute each optional embodiment of the above method, and the computing device can be a terminal, or a chip or chip system inside the terminal. Figure 4 As shown, the computing device 900 includes: a processor 910 , a memory 920 , and a communication interface 930 .

[0107] It should be understood that Figure 4 The communication interface 930 in the computing device 900 shown may be used to communicate with other devices, and may specifically include one or more transceiver circuits or interface circuits.

[0108] The processor 910 may be connected to a memory 920. The memory 920 may be used to store the program code and data. Therefore, the memory 920 may be a storage unit inside the processor 910, or an external storage unit independent of the processor 910, or a component including a storage unit inside the processor 910 and an external storage unit independent of the processor 910.

[0109] Optionally, the computing device 900 may further include a bus. The memory 920 and the communication interface 930 may be connected to the processor 910 via the bus. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 A line without an arrow is used to represent the bus, but this does not mean that there is only one bus or one type of bus.

[0110] It should be understood that in the embodiment of the present application, the processor 910 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 910 may adopt one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present application.

[0111] The memory 920 may include a read-only memory and a random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include a nonvolatile random access memory. For example, the processor 910 may also store information on the device type.

[0112] When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform any operation step of the above method and any optional embodiment thereof.

[0113] It should be understood that the computing device 900 according to the embodiment of the present application can correspond to the corresponding subjects in the methods according to the embodiments of the present application, and the above-mentioned and other operations and / or functions of each module in the computing device 900 are respectively for implementing the corresponding processes of each method of the present embodiment. For the sake of brevity, they will not be repeated here.

[0114] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0119] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A target following method for a spherical camera, characterized in that: The target is a gantry crane grab bucket; The method comprises: The spherical camera is used to obtain an image at a current viewing angle in real time, wherein the image includes the gantry crane grab bucket; Identify the center point of the gantry crane grab bucket position in the image by using a target detection model; Obtaining the current movement direction of the gantry crane grab bucket; Determine whether the center point of the gantry crane grab bucket is within the correct range frame according to the current movement direction of the gantry crane grab bucket and the current viewing angle of the spherical camera; According to the judgment result, the dome camera is controlled to rotate or pause.

2. The method according to claim 1, characterized in that The dome camera is installed below the cab of the gantry crane and faces the grab bucket side of the gantry crane.

3. The method according to claim 1, characterized in that The training method of the target detection model includes: Collecting images of working scenes that may include the gantry crane grab to obtain an original data set; Manually screening the original data set, removing images where the gantry crane grab bucket does not appear, and marking the range of the gantry crane grab bucket in the remaining images by selecting a box to obtain a marked data set; Dividing the labeled data set into a training set and a test set according to a certain ratio; The target detection model is trained using the training set, and the target detection model is verified using the test set.

4. The method according to claim 3, characterized in that The target detection model can re-collect image data during use to continue training the target detection model.

5. The method according to any one of claims 3 to 4, characterized in that: The center point (X0, Y0) of the gantry crane grab position is calculated according to the identification frame screened out after the maximum value suppression process, as follows: Among them, X0 and Y0 are the X-axis and Y-axis coordinates of the center point of the gantry crane grab bucket position, X1 and Y1 are the X-axis and Y-axis coordinates of the upper left vertex of the gantry crane grab bucket identification frame, and X2 and Y2 are the X-axis and Y-axis coordinates of the lower right vertex of the gantry crane grab bucket identification frame.

6. The method according to any one of claims 3 to 4, characterized in that: If the current movement direction is to the right relative to the current viewing angle, the correct range frame is a range frame with the center of the left half image of the current viewing angle as the center point; if the current movement direction is to the left relative to the current viewing angle, the correct range frame is a range frame with the center of the right half image of the current viewing angle as the center point; if the gantry crane grab bucket has not moved, the correct range frame is a range frame with the center of the current viewing angle image as the center point; If the center point of the gantry crane grab bucket is within the correct range frame, the spherical camera does not need to be rotated; if the center point of the gantry crane grab bucket is not within the correct range frame, the spherical camera needs to be rotated, and the rotation time of the spherical camera is calculated by selecting a rotation speed of the spherical camera; The rotation time of the dome camera is calculated according to the following formula: X0-X′0=V x t x ,Y0-Y′0=V y t y Among them, X′0, Y0′ is the coordinate of the center point of the correct range frame, V x ,V y is the rotation speed in the x-axis and y-axis directions, t x ,t y is the rotation time in the x-axis and y-axis directions.

7. The method according to claim 6, characterized in that The control system has a plurality of speed gears and can use different rotation speeds according to the position of the gantry crane grab in the current viewing angle.

8. The method according to claim 6, characterized in that After the rotation and pause movement is completed, the target detection and calculation are performed again to determine whether the set monitoring angle of view, that is, the correct range frame, is reached; if not, the steps of calculating, sending control instructions, and controlling movement are continued; if it is reached, the movement of the dome camera is stopped and wait for the next monitoring cycle.

9. A target following device for a spherical camera, characterized in that: include: A data collection module, used to obtain an image at a current viewing angle in real time through a dome camera, wherein the image includes a gantry crane grab bucket; The data processing module identifies the center point of the gantry crane grab bucket in the image through the target detection model; obtains the current movement direction of the gantry crane grab bucket; and determines whether the center point of the gantry crane grab bucket is within the correct range frame according to the current movement direction of the gantry crane grab bucket and the current viewing angle of the dome camera; The motion control module is used to control the spherical camera to rotate or pause according to the judgment result.

10. A computing device, characterized in that include: processor, and A memory having program instructions stored thereon, wherein when the program instructions are executed by the processor, the processor executes the target tracking method for a spherical camera according to any one of claims 1 to 8.

Images