Lifting Detection Method for Container Truck, Container Crane and Related Devices

Abstract

An embodiment of this specification provides a lifting detection method for a container truck, a container crane, and related devices. The method includes: identifying a specified target object included in the container truck from a reference frame image collected for the container truck; respectively performing scaling processing on a plurality of target objects identified in subsequent frame field images to obtain a plurality of scaled target objects; determining the specified target object among the plurality of scaled target objects to obtain the motion trajectory of the specified target object; and performing lifting detection according to the motion trajectory of the specified target object. The accuracy of lifting detection for the container truck during operation can be improved to a certain extent.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of container truck operation safety, and particularly to a lifting detection method for container trucks, a container crane, and related devices. Background Art

[0002] The lifting detection of container trucks can be used to determine whether a container truck is lifted together with a container by a container crane during the operation of the container truck yard bridge by using the continuous video images collected by the image acquisition devices installed on site, so as to prevent the container truck from being mistakenly lifted and causing safety accidents.

[0003] Currently, usually, according to the collected video images, object tracking is performed on the objects on the container truck, such as the vehicle frame, tires, etc., and according to the motion trajectories of the extracted target objects, it is determined whether the container truck has a lifting condition.

[0004] However, in some scenarios of high-speed operation, such as when the lifting speed of the container truck in the yard bridge is relatively fast, it is easy to lose or mis-track the target during the process of extracting the motion trajectory, resulting in inaccurate lifting detection. Summary of the Invention

[0005] In view of this, the embodiments of this specification are committed to providing a lifting detection method for container trucks, a container crane, and related devices, which can improve the accuracy of lifting detection for container trucks during operation to a certain extent.

[0006] Multiple embodiments in this specification provide a lifting detection method for container trucks, and the method includes: identifying a specified target object included in the container truck from the reference frame image collected for the container truck; respectively performing scaling processing on multiple target objects identified in the subsequent frame on-site images to obtain multiple scaled target objects; wherein, the specified target object is included in the multiple target objects, determining the specified target object among the multiple scaled target objects to obtain the motion trajectory of the specified target object; and performing lifting detection according to the motion trajectory of the specified target object.

[0007] Optionally, each of the multiple target objects corresponds to a preset scaling ratio; wherein, the corresponding preset scaling ratio is determined according to multiple dimensional features of the target object; the step of respectively performing scaling processing on multiple target objects identified in the subsequent frame on-site images to obtain multiple scaled target objects includes: respectively scaling the multiple target objects according to the corresponding preset scaling ratios to obtain multiple scaled target objects.

[0008] Optionally, at least the target object size and the target object position are included in the multiple dimensional features.

[0009] Optionally, the preset scaling ratio includes a width scaling ratio and a height scaling ratio; wherein, the height scaling ratio is greater than the width scaling ratio.

[0010] Optionally, the step of determining the specified target object among multiple scaling target objects to obtain the motion trajectory of the specified target object includes: obtaining a predicted region of the specified target object in the subsequent frame live image; performing a correlation matching operation between each of the multiple scaling target objects and the predicted region to determine the specified target object according to the matching result.

[0011] Optionally, in the step of determining the specified target object among multiple scaling target objects to obtain the motion trajectory of the specified target object, it further includes: performing a scaling process on the predicted region to obtain a scaled predicted region; correspondingly, performing a correlation matching operation between each of the multiple scaling target objects and the scaled predicted region to determine the specified target object according to the operation result.

[0012] Optionally, the step of performing a correlation matching operation between each of the multiple scaling target objects and the scaled predicted region to determine the specified target object according to the operation result includes: calculating the intersection over union (IoU) between each of the multiple scaling target objects and the scaled predicted region to obtain corresponding IoU results; determining the scaling target object with the largest IoU with the scaled predicted region among the multiple scaling target objects as the specified target object. One embodiment of this specification provides a lifting detection device for a container truck, where the lifting detection device includes: an identification module configured to identify a specified target object included in the container truck from a reference frame image collected for the container truck; a scaling module configured to perform a scaling process on each of multiple target objects identified in a subsequent frame live image to obtain multiple scaled target objects; a determination module configured to determine the specified target object among the multiple scaled target objects to obtain the motion trajectory of the specified target object; and a lifting detection module configured to perform a lifting detection according to the motion trajectory of the specified target object.

[0013] One embodiment of this specification provides a container crane, where the container crane includes: a lifting detection device for a container truck configured to execute the method in any of the foregoing embodiments, and a container crane body; the lifting detection device for the container truck is installed on the container crane body.

[0014] One embodiment of this specification provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the method in any of the foregoing embodiments when executing the computer program.

[0015] One embodiment of this specification provides a computer storage medium storing computer program instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0016] Multiple embodiments provided in this specification can perform scaling processing on multiple target objects identified in subsequent frame live images respectively to obtain multiple scaled target objects, and determine a specified target object identified in the reference frame image among the multiple scaled target objects to achieve target tracking. Furthermore, lifting detection can be performed based on the motion trajectory of the obtained specified target object. In this way, by scaling multiple target objects identified in the live image during target tracking and then determining the specified target object, the target tracking for the specified target object and the obtained motion trajectory can be made more accurate, thereby improving the accuracy of lifting detection. To a certain extent, it can solve the problem of inaccurate lifting detection caused by losing or following the wrong target in the scenario of high-speed operation of container truck yard cranes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the lifting detection method for a container truck provided in one embodiment of this specification.

[0018] Figure 2 It is a schematic diagram of the installation position of the image acquisition device provided in one embodiment of this specification.

[0019] Figure 3 It is a schematic diagram of the lifting detection process for a container truck provided in one embodiment of this specification.

[0020] Figure 4 It is a schematic diagram of the lifting detection device for a container truck provided in one embodiment of this specification.

[0021] Figure 5 It is a schematic diagram of a computer device provided in one embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0023] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the embodiments of the present application, "a plurality of" means two or more unless otherwise specifically defined.

[0024] Please refer to Figures 1 - 3 . One embodiment of this specification provides a lifting detection method for a container truck. The lifting detection method for the container truck may include the following steps.

[0025] Step S110: Identify a specified target object included in the container truck from the reference frame image collected for the container truck.

[0026] In this embodiment, the reference frame image may be used as the initial frame in the image sequence during the container lifting operation of the container truck for target tracking analysis in subsequent frames. Specifically, the reference frame image may include the video image collected at the start of the lifting operation. For example, the video image collected when the spreader starts to lift the container may be used as the reference frame image.

[0027] In this embodiment, the reference frame image and subsequent video images may be collected by image acquisition devices arranged on site. Specifically, the image acquisition devices may be arranged on both sides of the container truck. For example, a plurality of image acquisition devices may be arranged on the sea side and the land side respectively to collect video images of the container truck during operation from multiple angles. Of course, in actual operation, considering the limitations of the quay crane and the yard, it is a relatively common solution in the art to arrange the image acquisition devices on the same side of the container truck. For example, a plurality of image acquisition devices may be arranged on the bottom bracket of the rubber-tyred gantry crane. The image acquisition devices may specifically include cameras, cameras, etc. Specifically, for example, a plurality of image acquisition devices may be arranged on the bottom bracket of the rubber-tyred gantry crane. The image acquisition devices may specifically include cameras, cameras, etc. Specifically, as Figure 3 shown, 4 cameras may be installed on the crossbeam on the operation side of the quay crane container truck. Among them, 2 cameras for observing 20-foot containers may be set at a distance of 2 meters to 4 meters from the center of the quay crane, and 2 cameras for observing 40-foot and 45-foot containers may be set at a distance of 5 meters to 7 meters from the center of the quay crane. The 4 cameras may be set at a distance of 1.6 meters - 1.8 meters from the ground. The camera angles may be slightly tilted downward at a certain angle to ensure that the parts below 0.5 meters - 1 meter high of the vehicle frame and the container can be captured. For example, it may be tilted downward by 5° - 20°.

[0028] In this embodiment, the specified target object can be used as an analysis object for determining whether the container truck has been lifted. The movement of the container truck can be analyzed by analyzing the movement trajectory of the specified target object in the collected image sequence, and then it can be determined whether there is a lift. Specifically, the specified target object can be an object or vehicle component on the container truck. For example, the specified target object can be a light source, the container truck itself, the vehicle frame, the tires, the container on the vehicle, etc.

[0029] In this embodiment, from the reference frame image collected for the container truck, the specified target object included in the container truck is identified. Based on a target detection algorithm, such as methods like sliding window, convolutional neural network, etc., the target of interest in the reference frame image is identified to determine the specified target object.

[0030] In some embodiments, it is also possible to identify and classify multiple targets of interest in the reference frame image based on a multi-class target detection algorithm, and use the multiple targets as multiple specified target objects for lift analysis and detection, and judge whether there is a lift by synthesizing the analysis results of the movement trajectories of the multiple specified target objects.

[0031] Step S120: Perform scaling processing on the multiple target objects identified in the subsequent frame live image respectively to obtain multiple scaled target objects; wherein, the multiple target objects include the specified target object.

[0032] In some cases where the container truck needs to perform high-speed operations, such as when an internal container yard bridge is operating, the speed of lifting the container with a spreader may be relatively fast. Therefore, the position and size of the specified target object in the subsequent frame live image may change significantly. For example, the position may deviate a lot, and due to the distance from the image acquisition device being far, the size may shrink, making it difficult to find a target object in the subsequent frame that matches the specified target object in the reference frame when performing target tracking on the specified target object in the reference frame. Or when there are many similar target objects of the same type near the specified target object, mis-matching is likely to occur, resulting in losing or mis-tracking the target, making the movement trajectory obtained by tracking the specified target object inaccurate, and thus affecting the accuracy of lift detection. In this embodiment, by performing scaling processing on the multiple target objects identified in the subsequent frame live image respectively, it is possible to increase the matching probability of the specified target object in the reference frame image and the subsequent frame live image in the case of high-speed operation of the container truck, reduce the probability of incorrect tracking, improve the accuracy of the movement trajectory of the obtained specified target object, and thus improve the accuracy of lift detection.

[0033] In this embodiment, the subsequent frame live image can be continuously acquired by the image acquisition device of the reference frame image. Specifically, for example, multiple image acquisition devices arranged on the bottom bracket of the rubber-tyred gantry crane can continuously acquire the subsequent operation images of the container truck after acquiring the reference frame image. Of course, in some cases, the position of the image acquisition device can also be adjusted before continuously acquiring the subsequent live image. Specifically, for example, when the specified target object is located at the edge position of the reference frame image, the position or angle of the image acquisition device can be adjusted so that the specified target object will not fail to be tracked and recognized because it does not appear in the subsequently acquired live image.

[0034] In this embodiment, the subsequent frame live image can include one frame of image after the reference frame image, such as an adjacent frame image, or can also include multiple consecutive frames of images after the reference frame image, so as to be used together with the reference frame image to form a video image sequence of the container truck yard crane operation. Specifically, the video image sequence can be used to represent the overall operation process of the container truck from the start of operation to the container being lifted and the container truck driving away.

[0035] In this embodiment, the multiple target objects can be used as candidate objects for matching with the specified target object in the reference frame in the subsequent frame live image, so as to determine the specified target object among the multiple target objects, and then realize the tracking of the specified target object in the subsequent frame live image. Specifically, the target object can include the objects on the container truck identified by performing target detection on the subsequent frame live image, or can also include multiple similar target objects identified in the subsequent frame live image with reference to the position, size and state of the specified target object in the reference frame image.

[0036] In this embodiment, scale processing is respectively performed on the multiple target objects identified in the subsequent frame live image to obtain multiple scaled target objects. The scale ratio can be determined according to the acquisition time difference between the subsequent frame live image and the reference frame image, and then the multiple target objects are scaled according to the scale ratio. It is also possible to scale the multiple target objects according to a unified preset scale ratio.

[0037] In some embodiments, when the subsequent frame live image includes multiple consecutive frames of images after the reference frame image, multiple target objects in each frame of the consecutive frames of images can be respectively identified, and then scale processing is respectively performed on the multiple target objects in each frame to obtain multiple scaled target objects in each frame.

[0038] In some embodiments, each of the multiple target objects corresponds to a preset scaling ratio; wherein, the corresponding preset scaling ratio is determined according to the multiple dimensional features of the target object; the steps of respectively performing scaling processing on the multiple target objects identified in the subsequent frame live image to obtain multiple scaled target objects include: for the multiple target objects, respectively scaling them according to the corresponding preset scaling ratios to obtain multiple scaled target objects.

[0039] In some embodiments, different scaling ratios can be preset according to the different object attributes of the target object. Specifically, the corresponding preset scaling ratio can be determined according to the multiple dimensional features of the target object, and the multiple dimensional features can be used to characterize the object attributes of the target object. Specifically, it can include the size of the target object, such as width, height, etc., and can also include the position of the target object. Among them, the position of the target object can include the position coordinates of the target object in the subsequent frame live image, can also include the relative position between the target object and the object with a relatively high similarity identified, and can also include the distribution pattern of multiple target objects in the subsequent frame live image. Specifically, for example, since the locks on the container truck may be small in the image and the distance between the locks is small, in order to prevent the occurrence of target tracking errors, the size of the locks, the position of the locks in the image, and the relative position between the locks can be considered to determine the preset scaling ratio. Of course, in some embodiments, the multiple dimensional features can also include the type of the target object, its function in the container truck operation, and its historical movement trajectory, etc. For example, by analyzing the historical movement trajectory of the target object in the case of the container truck's historical operation, the probability distribution of the position and state of the target object in the subsequent frame live image can be understood, and the scaling ratio can be reasonably set according to the likelihood.

[0040] In some embodiments, the preset scaling ratio can be set for the target objects on different container trucks respectively according to historical data and experience summary, or after further testing and verification.

[0041] In some embodiments, the preset scaling ratio includes a width scaling ratio and a height scaling ratio; wherein, the height scaling ratio is greater than the width scaling ratio.

[0042] During the operation of the gantry crane at the container yard, the detection of the container truck lifting usually targets the process from when the spreader is ready to lift the container to when the container truck is ready to drive out of the gantry crane area. It can be understood that during this process, whether it is the container truck, the objects on the container truck, the container or the spreader, their movement in the horizontal direction is relatively small. Therefore, for the target object in the subsequent frame of the on-site image, compared with the reference frame image, its position change is more in the vertical direction. By scaling the target object in height, the probability of correctly identifying the specified target object during target tracking can be increased. Since the movement of the target object in the horizontal direction is relatively small, if the width is scaled at the same ratio as the height scaling, the probability of misidentifying similar or identical target objects will increase. Therefore, by setting a width scaling ratio that is relatively smaller than the height scaling ratio, the accuracy of target tracking and recognition after scaling the target object based on the preset scaling ratio can be further improved, and the probability of losing the target and misidentifying can be reduced.

[0043] In some embodiments, different target objects can correspond to different width scaling ratios and height scaling ratios. Similarly, the specific ratios can be determined according to the attributes of the target objects. Specifically, for example, for the lock on the container truck, it can be set to magnify the width by 2 pixels and magnify the height by 3 pixels.

[0044] Step S130: Determine the specified target object among the multiple scaled target objects to obtain the movement trajectory of the specified target object.

[0045] In this embodiment, based on the target tracking algorithm, the specified target object can be determined among the multiple scaled target objects to obtain the movement trajectory of the specified target object. Among them, the target tracking algorithm can be used to automatically identify and follow the specified target object in a continuous video image sequence, and achieve target tracking by updating the position and state of the target object in the subsequent frame of the on-site image. Specifically, the target tracking algorithm can include Kalman Filter, KCF, Deep SORT, etc.

[0046] In this embodiment, the movement trajectory of the specified target object can represent the position change law and movement trend of the specified target object during the entire container lifting operation, so as to be used to analyze the movement pattern of the specified target object, and further serve as the basis for judging whether the container truck is lifted. Specifically, the movement trajectory of the specified target object can be determined according to the position and state of the specified target object in the reference frame image and the subsequent frame of the on-site image, and specifically can include time and position information, the shape and direction of the movement trajectory, and the movement state information of the specified target object, such as speed, acceleration, etc.

[0047] In this embodiment, the process of determining the specified target object among multiple scaling target objects to obtain the motion trajectory of the specified target object may include: based on one or more feature points of the specified target object obtained from a reference frame, performing feature point matching in subsequent frames, and then performing motion estimation according to the displacement of the feature points in the subsequent frames, so as to obtain the motion trajectory of the specified target object.

[0048] In some embodiments, to determine the specified target object among multiple scaling target objects to obtain the motion trajectory of the specified target object, a neural network model can be trained to learn the features of the target object, specifically including appearance features and motion features. The trained model is used to extract the features of the specified target object in the reference frame image and the features of multiple target objects in subsequent frames, and then the features of multiple target objects are respectively subjected to feature matching with the features of the specified target object and a similarity score is calculated. The target object with the highest similarity is used as the specified target object in the subsequent frame, so as to obtain the motion trajectory of the specified target object.

[0049] In some embodiments, when the subsequent frame live image includes a series of consecutive frames after the reference frame image, for each frame in the series of consecutive frame live images, the specified target object is determined among the multiple target objects identified in each frame of the live image. In this way, the tracking of the specified target object can be realized in the series of consecutive frame live images, so as to obtain its motion trajectory according to the position and status information of the specified target object in each frame of the live image.

[0050] In some embodiments, the steps of determining the specified target object among multiple scaling target objects to obtain the motion trajectory of the specified target object may further include: obtaining the prediction region of the specified target object in the subsequent frame live image; performing an association degree matching operation between multiple scaling target objects and the prediction region respectively, so as to determine the specified target object according to the matching result.

[0051] In some embodiments, the prediction region can be used to predict the position of the specified target object in the subsequent frame live image. Specifically, according to the position and motion status information of the specified target object in the reference frame, its position and status in the subsequent frame can be predicted. To obtain the prediction region of the specified target object in the subsequent frame live image, Kalman filtering or some motion models can be used to predict the specified target object in the reference frame to obtain the prediction region.

[0052] In some embodiments, the relevance matching operation can be used to calculate the relevance or similarity between the scaled target object and the prediction region, or can also be understood as representing the relevance or similarity between the target object in the subsequent frame and the specified target object. Specifically, for example, the relevance matching operation can include methods for measuring the similarity of feature vectors, or can also include calculating the intersection over union (IoU) between the detection box of the scaled target object and the prediction box of the prediction region.

[0053] In some embodiments, according to the matching result after the relevance matching operation, the similarities can be sorted, and the scaled target object with the highest similarity to the prediction region is used as the specified target object. Furthermore, the target object corresponding to the scaled target object can be determined as the specified target object in the subsequent frame, so as to achieve target association of the specified target object in different frames.

[0054] In some embodiments, in the step of determining the specified target object among multiple scaled target objects to obtain the motion trajectory of the specified target object, it further includes: performing a scaling process on the prediction region to obtain a scaled prediction region; correspondingly, performing a relevance matching operation between multiple scaled target objects and the scaled prediction region respectively, so as to determine the specified target object according to the operation result.

[0055] In some embodiments, in order to further improve the accuracy of target association matching for the specified target object in the reference frame image and the subsequent frame live image, the prediction region can also be first scaled and then a relevance matching operation is performed to determine the specified target object according to the operation result.

[0056] In some embodiments, the step of performing a relevance matching operation between multiple scaled target objects and the scaled prediction region respectively to determine the specified target object according to the operation result can include: calculating the intersection over union between multiple scaled target objects and the scaled prediction region respectively to obtain corresponding intersection over union results; among multiple scaled target objects, the scaled target object with the largest intersection over union with the scaled prediction region is determined as the specified target object.

[0057] Step S140: Perform a lifting detection according to the motion trajectory of the specified target object.

[0058] In this embodiment, based on the motion trajectory of the specified target object, by analyzing the position movement pattern of the specified target object, it can be determined whether the container truck has been lifted. Specifically, for example, the specified target object can be the vehicle frame. When the vertical component of the motion trajectory of the vehicle frame starts from the reference frame image and has an upward movement exceeding the specified threshold, such as an upward movement of 2 pixels, it can be determined that the container truck has been lifted.

[0059] In some embodiments, based on the motion trajectory of the specified target object, by analyzing the change in the motion state of the specified target object, it can also be determined whether the container truck has been lifted. Specifically, for example, when the vertical component of the speed of the specified target object in the subsequent frame of the on-site image exceeds the specified threshold, it can be determined that the container truck has been lifted.

[0060] In some embodiments, in combination with the specific situation, based on the motion trajectory of the specified target object, more perfect analysis and judgment rules can also be established. Specifically, for example, in some cases, considering the positions of the quay crane and the yard, the image acquisition device for collecting the on-site images of the container lifting operation is usually set on one side of the container truck. For example, it can be set at the crossbeam on the quay crane lane side. This results in a limited range of the image data that the image acquisition device can collect, and there are certain shooting blind spots. For example, if the image acquisition device is set on the left side of the driving direction of the container truck, then due to the occlusion of the container truck and the container on the vehicle, it may be difficult to capture the picture on the right side of the driving direction of the container truck. Therefore, in the case where there are blind spots in the shooting range of the image acquisition device, if a container truck is lifted in the blind area, it is difficult to directly judge by the way of collecting images through the image acquisition device. By analyzing and judging the possible lifting situation in the shooting field of view blind area by using the object motion trajectory of the target object in the subsequent on-site images collected within the shooting range of the image acquisition device, the lifting judgment rule of the container truck can be made more perfect, thereby improving the accuracy of lifting detection.

[0061] Specifically, in the above case, if a lifting operation occurs in the shooting blind area of the image acquisition device, such as the lock head not being unlocked, the container truck will be lifted by the spreader on one side of the shooting blind area. It can be understood that at this time, since the upward pulling force on the container truck is uneven, the side where the lifting does not occur is not directly subjected to the upward pulling force, but will be lifted together with the pulling force received on the shooting blind area side. Considering the force application point and the center of gravity position, if a lift occurs on one side of the shooting blind area of the image acquisition device, then on the other side, that is, the side of the container truck shown in the on-site image collected by the image acquisition device, the container truck will move downward to a certain extent to maintain the overall balance of the container truck when it is lifted. Therefore, by tracking the object motion trajectory of the specified target object, if there is a downward vertical component exceeding the specified threshold in the object motion trajectory, it can be determined that the container truck has been lifted. Specifically, it can be understood that if there is a downward vertical component exceeding the specified threshold in the vertical component of each trajectory point in the object target trajectory of the specified target object, it can be determined that the container truck has been lifted. Therefore, in the step of performing lift detection according to the motion trajectory of the specified target object, it may further include: if there is a downward vertical component exceeding the specified threshold in the motion trajectory of the specified target object, it can be determined that the container truck has been lifted.

[0062] Please refer to Figure 4 The embodiment of this specification also provides a lift detection device for a container truck, which is characterized in that the device includes: an identification module for identifying a specified target object included in the container truck from a reference frame image collected for the container truck; a scaling module for respectively performing scaling processing on a plurality of target objects identified in subsequent frame on-site images to obtain a plurality of scaled target objects; wherein, the specified target object is included in the plurality of target objects; a determination module for determining the specified target object among the plurality of scaled target objects to obtain the motion trajectory of the specified target object; a lift detection module for performing lift detection according to the motion trajectory of the specified target object.

[0063] For the specific explanation of the lift detection device for the container truck, reference can be made to the above-mentioned embodiments for comparison and explanation, which will not be elaborated here.

[0064] The embodiment of this specification also provides a container crane, which includes: a lift detection device for a container truck that executes the method described in any of the foregoing embodiments, and a container crane body; wherein, the lift detection device for the container truck is installed on the container crane body.

[0065] The embodiments of this specification also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the computer is caused to execute the lifting detection method of the container truck in any of the above embodiments.

[0066] The embodiments of this specification also provide a computer program product containing instructions. When the instructions are executed by a computer, the computer is caused to execute the lifting detection method of the container truck in any of the above embodiments.

[0067] Please refer to Figure 5 The embodiments of this specification can provide a computer device, which includes: a memory, and one or more processors communicatively connected to the memory; instructions executable by the one or more processors are stored in the memory, and when the instructions are executed by the one or more processors, the one or more processors are caused to implement the method in any of the above embodiments.

[0068] In some embodiments, the computer device may include a processor, a non-volatile storage medium, an internal memory, a communication interface, a display device, and an input device connected by a system bus. The non-volatile storage medium may store an operating system and related computer programs.

[0069] It can be understood that the specific examples herein are only for helping those skilled in the art to better understand the embodiments of this specification, rather than limiting the scope of the present invention.

[0070] It can be understood that in the various embodiments of this specification, the magnitudes of the sequence numbers of the processes do not mean the order of execution. The order of execution of the processes should be determined according to their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of this specification.

[0071] It can be understood that the various embodiments described in this specification can be implemented alone or in combination, and this specification does not limit this.

[0072] As mentioned above, the above are only the specific embodiments of this specification, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this specification, and all should be covered by the protection scope of this specification. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A lifting detection method for a container truck, characterized in that, The method includes: Identifying a specified target object included in the container truck from a reference frame image collected for the container truck; Performing scaling processing on multiple target objects identified in subsequent frame live images respectively to obtain multiple scaled target objects; wherein, the specified target object is included in the multiple target objects; Determining the specified target object among the multiple scaled target objects to obtain the motion trajectory of the specified target object; Performing lifting detection according to the motion trajectory of the specified target object; The step of determining the specified target object among the multiple scaled target objects to obtain the motion trajectory of the specified target object includes: Obtaining a predicted region of the specified target object in the subsequent frame live image; Performing scaling processing on the predicted region to obtain a scaled predicted region; Performing a correlation matching operation on the multiple scaled target objects respectively with the scaled predicted region to determine the specified target object according to the operation result.

2. The method according to claim 1, wherein Each of the multiple target objects corresponds to a preset scaling ratio; wherein, the corresponding preset scaling ratio is determined according to multiple dimensional features of the target object; the step of performing scaling processing on multiple target objects identified in subsequent frame live images respectively to obtain multiple scaled target objects includes: Performing scaling on the multiple target objects respectively according to the corresponding preset scaling ratios to obtain multiple scaled target objects.

3. The method according to claim 2, wherein At least the target object size and the target object position are included in the multiple dimensional features.

4. The method according to claim 3, wherein The preset scaling ratio includes a width scaling ratio and a height scaling ratio; wherein, the height scaling ratio is greater than the width scaling ratio.

5. The method according to claim 1, characterized in that, The step of performing a correlation matching operation on the multiple scaled target objects respectively with the scaled predicted region to determine the specified target object according to the operation result includes: Calculating the intersection over union (IoU) between the multiple scaled target objects and the scaled predicted region respectively to obtain corresponding IoU results; Determining the scaled target object with the largest IoU between it and the scaled predicted region among the multiple scaled target objects as the specified target object.

6. A lifting detection device for a container truck, characterized in that, The device includes: An identification module, configured to identify a specified target object included in the container truck from a reference frame image collected for the container truck; A scaling module, configured to perform scaling processing on multiple target objects identified in subsequent frame live images respectively to obtain multiple scaled target objects; A determination module, configured to determine the specified target object among the multiple scaled target objects to obtain the motion trajectory of the specified target object; the steps include: obtaining a predicted region of the specified target object in the subsequent frame live image; performing scaling processing on the predicted region to obtain a scaled predicted region; performing a correlation matching operation on the multiple scaled target objects respectively with the scaled predicted region to determine the specified target object according to the operation result; A lifting detection module, configured to perform lifting detection according to the motion trajectory of the specified target object.

7. A computer instruction storage medium, characterized in that, The computer storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method described in any one of claims 1-5 is implemented.

8. A container crane, characterized in that, The container crane includes: a lifting detection device for a container truck that executes the method described in any one of claims 1-5, and a container crane body; the lifting detection device for the container truck is installed on the container crane body.

Images