Cage sliding detection method and device, computer equipment, and storage medium

Abstract

The present application relates to a method and device for detecting the sliding of a material cage, a computer device, and a storage medium. The method comprises: obtaining a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup; marking the material cage image as the image to be tested, or extracting and processing the material cage video stream to obtain the image to be tested; detecting the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup; judging whether the coordinates of the foot cup are within a preset coordinate range, and obtaining the sliding detection result of the material cage to be tested. The present method can realize the sliding detection of the material cage.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a cage sliding detection method and device, computer equipment, and storage medium. Background Art

[0002] Forklifts, a crucial tool in modern logistics and transportation, are wheeled transport vehicles used for loading and unloading, stacking, short-distance transport, and heavy-lifting of palletized cargo. They are widely used in ports, stations, airports, freight yards, factory workshops, warehouses, distribution centers, and distribution centers. They can also enter ship cabins, train compartments, and containers to load, unload, and carry palletized cargo, making them essential equipment for pallet and container transportation.

[0003] When a forklift is transporting a material cage, uneven surfaces can easily transmit vibrations to the cage, causing it to slip on the forks and eventually fall off the forks. Therefore, how to detect cage slippage is a technical problem that technicians in this field urgently need to solve. Summary of the Invention

[0004] Based on this, it is necessary to provide a cage sliding detection method and device, computer equipment, and storage medium that can realize sliding detection of the cage in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting cage sliding. The method comprises:

[0006] Acquire a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup;

[0007] Marking the cage image as the image to be tested, or extracting and processing the cage video stream to obtain the image to be tested;

[0008] Detecting the image to be tested according to a preset sliding detection model to obtain position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup;

[0009] It is determined whether the foot cup coordinates are within a preset coordinate interval to obtain a sliding detection result of the material cage to be tested.

[0010] In one embodiment, the cage image or the cage video stream is obtained by photographing the cage to be tested with a camera pre-installed on a forklift, and a display interface corresponding to the cage image or the cage video stream is provided with shooting auxiliary lines and an area of interest;

[0011] Before obtaining the cage image or cage video stream obtained by photographing the cage to be tested, the method further includes:

[0012] The angle of the camera is adjusted so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is located in the region of interest.

[0013] In one embodiment, the region of interest includes a first region of interest and a second region of interest, the shooting auxiliary lines include a first auxiliary line, a second auxiliary line and a third auxiliary line, the first auxiliary line intersects with the second auxiliary line and the third auxiliary line respectively, and the material cage to be tested includes two foot cups, respectively named as a first foot cup and a second foot cup;

[0014] The adjusting the angle of the camera so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is in the region of interest includes:

[0015] Adjusting the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is within the first region of interest;

[0016] The roll angle of the camera is adjusted so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

[0017] In one embodiment, detecting the cage image according to a preset sliding detection model to obtain position information of the foot cup in the cage image includes:

[0018] Inputting the image to be tested into a trained sliding detection model, wherein the sliding detection model is used to perform recognition processing on the image to be tested to obtain position information of the foot cup in the image to be tested;

[0019] The position information output by the sliding detection model is obtained.

[0020] In one embodiment, the sliding detection model is obtained by training the labeled original image data as input of a preset original detection algorithm model.

[0021] In one embodiment, determining whether the foot cup coordinates are within a preset coordinate interval to obtain the sliding detection result of the material cage to be tested includes:

[0022] If the foot cup coordinates are within the preset coordinate interval, no relative sliding occurs between the material cage to be tested and the fork of the forklift;

[0023] If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

[0024] In a second aspect, the present application further provides a cage sliding detection device. The device comprises:

[0025] An acquisition module is used to acquire a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup;

[0026] A processing module, configured to mark the cage image as an image to be tested, or extract and process the cage video stream to obtain the image to be tested;

[0027] an identification and detection module, configured to detect the image to be tested according to a preset sliding detection model to obtain position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup;

[0028] The sliding detection module is used to determine whether the foot cup coordinates are within a preset coordinate range and obtain the sliding detection result of the material cage to be tested.

[0029] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the cage sliding detection method of the first embodiment is implemented.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cage sliding detection method of the embodiment of the first aspect.

[0031] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the cage sliding detection method of the first embodiment.

[0032] The above-mentioned material cage sliding detection method and device, computer equipment, and storage medium obtain a material cage image or a material cage video stream obtained by photographing the material cage to be tested, wherein the material cage to be tested is placed on a forklift and includes a foot cup, and then the material cage image is marked as the image to be tested, or the material cage video stream is extracted and processed to obtain the image to be tested. The image to be tested is then detected according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested, and the position information includes the foot cup coordinates. Finally, it is determined whether the foot cup coordinates are within a preset coordinate interval to obtain the sliding detection result of the material cage to be tested. By determining whether the obtained foot cup coordinates are within the preset coordinate interval, it is possible to determine whether relative sliding occurs between the material cage to be tested and the forklift, which is beneficial to improving the safety of forklift operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 1 is a flow chart of a cage sliding detection method according to an embodiment;

[0034] Figure 2 Schematic diagram of the structure of a material cage to be tested in one embodiment;

[0035] Figure 3 is a schematic diagram of a display interface in one embodiment;

[0036] Figure 4 A schematic flow chart of a cage sliding detection method in another embodiment;

[0037] Figure 5 is a schematic diagram of a display interface in another embodiment;

[0038] Figure 6 is a schematic diagram of a display interface in another embodiment;

[0039] Figure 7 for Figure 1 A flow chart of the specific method of step 106;

[0040] Figure 8 for Figure 1 A flow chart of the specific method of step 108;

[0041] Figure 9 A schematic flow chart of a cage sliding detection method in another embodiment;

[0042] Figure 10 1 is a structural block diagram of a cage sliding detection device in one embodiment;

[0043] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] In one embodiment, Figure 1 As shown, a cage sliding detection method is provided, comprising the following steps:

[0046] Step 102 , obtaining a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup.

[0047] Among them, the forklift in step 102 can be an unmanned forklift or a manned forklift. The material cage to be tested is placed on the forks of the forklift. A camera is set on the forklift to shoot the material cage to be tested, so as to obtain a material cage image corresponding to the material cage to be tested.

[0048] like Figure 2 As shown, in some embodiments, the present application provides a schematic structural diagram of a material cage to be tested. The material cage to be tested 202 includes a foot cup 204.

[0049] It should be noted that the camera can be installed at the root of the forklift's fork arm or other locations on the forklift. The camera is connected to the forklift's industrial computer via a network cable. The forklift's industrial computer stores the camera's IP address and port number to transmit the image or video stream captured by the camera to the industrial computer. Multiple cameras can be set, or one can be set. If two cameras are set, the two cameras can be set at the left and right ends of the fork arm root respectively, and the camera lenses at the left and right ends are set opposite to each other. The left end camera faces the direction of the right end camera, and the right end camera faces the direction of the left end camera. They are respectively used to obtain the cage images corresponding to the front and rear foot cups on the opposite side of the cage to be tested, so as to monitor whether the foot cups of the cage to be tested slide, thereby realizing the sliding detection of the cage. Among them, the root of the clamp arm refers to the end of the forklift's fork arm facing the forklift body.

[0050] Step 104 : Mark the cage image as the image to be tested, or extract and process the cage video stream to obtain the image to be tested.

[0051] Specifically, in step 104 of some embodiments, when the camera captures an image of the cage to be tested, the captured cage image is directly marked as the image to be tested. When the camera captures a video stream of the cage to be tested, the captured cage video stream is directly extracted and processed to obtain the image to be tested.

[0052] Step 106 : Detect the image to be tested according to a preset sliding detection model to obtain position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup.

[0053] In step 106, the sliding detection model is trained by using the labeled original image data as input to a preset original detection algorithm model. The original detection algorithm model can adopt the YOLOV5 algorithm model. The specific process is as follows:

[0054] First, a large amount of collected original image data is annotated using an image annotation tool (such as labelImg) to mark out the foot cup in the original image data.

[0055] Then the original image data and the original image data after the foot cup is annotated are input into the YOLOV5 model for training, and the training generates a .pt (the format of the model trained by yolov5 based on pytorch.pt) model, that is, a sliding detection model is obtained.

[0056] After obtaining the sliding detection model, it needs to be input into the industrial computer for execution to output the position information of the foot cup in the image to be tested, so as to facilitate the subsequent sliding detection of the cage.

[0057] The .pt model is converted into an onnx file format, which is then converted into a model file supported by the OpenVino inference engine. The resulting model file is then imported into an industrial computer for execution, enabling cage slip detection. The slip detection model can also be executed by a server, which acquires the image information to be tested, processes it, and outputs the position information of the foot cup in the image to the forklift, thereby also enabling cage slip detection.

[0058] It should be noted that the original image data can be obtained by shooting with a camera or obtained from a server, but each original image data must include the foot cup. If the original image data is obtained by shooting with a camera, the left and right cameras at both ends of the fork arm root will automatically capture images at regular intervals to obtain the original image data. Moreover, during the shooting process, in order to enrich the diversity of the original image data, the forklift needs to fork different material cages and change the scene more often, thereby improving the accuracy of the subsequent sliding detection of the material cage to be tested. At the same time, even when the material cage is in the sliding detection state, the left and right cameras at both ends of the forklift can continue to shoot the material cage to obtain the original image data.

[0059] Step 108 , determining whether the foot cup coordinates are within a preset coordinate range, and obtaining a sliding detection result of the material cage to be tested.

[0060] In step 108, based on the position information of the foot cup in the cage image obtained in the above steps, it is determined whether relative sliding occurs between the cage to be tested and the forklift, thereby obtaining a corresponding sliding detection result.

[0061] The above-mentioned material cage sliding detection method obtains a material cage image or a material cage video stream obtained by photographing the material cage to be tested, wherein the material cage to be tested is placed on a forklift and includes a foot cup, and then marks the material cage image as the image to be tested, or extracts and processes the material cage video stream to obtain the image to be tested. The image to be tested is then detected according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested, and the position information includes the foot cup coordinates. Finally, it is determined whether the foot cup coordinates are within a preset coordinate interval to obtain the sliding detection result of the material cage to be tested. By determining whether the obtained foot cup coordinates are within the preset coordinate interval, it is possible to determine whether relative sliding occurs between the material cage to be tested and the forklift, which is beneficial to improving the safety of forklift operation.

[0062] In some embodiments, the cage image or cage video stream is obtained by capturing the cage to be tested with a camera pre-installed on a forklift, and a display interface corresponding to the cage image or cage video stream is provided with shooting auxiliary lines and a region of interest. Prior to step 102, the cage slip detection method further includes the following steps:

[0063] Adjust the camera angle so that the shooting guide line passes through the center of the foot cup and the foot cup is within the area of interest.

[0064] In this embodiment, the display interface refers to the real-time display interface during sliding detection, which can display the real-time video stream captured by the camera or the captured cage image. The camera angle is adjusted so that the shooting auxiliary line passes through the center of the foot cup, and the foot cup is set in the area of interest. Through such a setting, it is possible to avoid the sliding detection model from identifying the foot cups of other cages, thereby improving the accuracy of cage sliding detection. Among them, the display interface can be the display interface of the camera, or the display interface of an industrial computer connected to the camera, or the display interface of other devices connected to the camera for communication, which receives the real-time video stream or cage image captured by the camera through the communication network and displays it. Other cages refer to other cages that are not placed on the forks.

[0065] In some embodiments, please refer to Figures 3 to 6 The region of interest includes a first region of interest 308 and a second region of interest 310. The shooting auxiliary lines include a first auxiliary line 302, a second auxiliary line 304, and a third auxiliary line 306. The first auxiliary line 302 intersects with the second auxiliary line 304 and the third auxiliary line 306 respectively. The material cage to be tested includes two foot cups, which are named as the first foot cup 2041 and the second foot cup 2042. Figure 3 This is a schematic diagram of the display interface before adjustment. Figure 5 and Figure 6 This is a schematic diagram of the display interface after adjustment according to the technical solution of this embodiment.

[0066] In some embodiments, the first auxiliary line 302 is perpendicular to the second auxiliary line 304 and the third auxiliary line 306 respectively, the third auxiliary line 306 is parallel to the second auxiliary line 304, the first auxiliary line 302 and the second auxiliary line 304 intersect at the center of the first area of interest 308, and the first auxiliary line 302 and the third auxiliary line 306 intersect at the center of the second area of interest 310.

[0067] The step of "adjusting the camera angle so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is within the area of interest" specifically includes:

[0068] Step 402 : Adjust the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is in the first region of interest.

[0069] Step 404 : Adjust the roll angle of the camera so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

[0070] Specifically, in this embodiment, the camera can capture the two foot cups of each material cage by setting the first area of interest 308 and the second area of interest 310, and by adjusting the yaw angle and pitch angle of the camera, so that the first auxiliary line 302 and the second auxiliary line 304 intersect at the center of the first foot cup 2041, and the first foot cup 2041 is set in the first area of interest 308; by adjusting the roll angle of the camera, so that the first auxiliary line 302 and the third auxiliary line 306 intersect at the center of the second foot cup 2042, and the second foot cup 2042 is set in the second area of interest 310, it is possible to avoid the sliding detection model from identifying the foot cups of other material cages, and increase the sliding detection model's attention to the foot cups of the material cage to be tested, thereby improving the accuracy of the cage sliding detection.

[0071] It should be noted that in this solution, adjusting the yaw and pitch angles of the camera to adjust the positions of the first auxiliary line 302 and the second auxiliary line 304 does not mean that the first auxiliary line 302 and the second auxiliary line 304 are adjustable. Instead, the positions of the auxiliary lines are adjusted by adjusting the angle of the camera. This is similar to using a camera to take a photo of a person, where a frame is preset and the person's head needs to be adjusted to fit within the frame. By adjusting the camera's shooting angle, the person's head can be adjusted to fit within the frame, rather than directly adjusting the position of the frame.

[0072] In some embodiments, as Figure 7 As shown, step 106 includes the following steps:

[0073] Step 702: Input the image to be tested into the trained slide detection model. The slide detection model is used to identify the cup in the image to be tested and obtain the position information of the cup in the image to be tested.

[0074] Step 704: Obtain the position information output by the sliding detection model.

[0075] Specifically, in this embodiment, the captured image to be tested is input into the sliding detection model so that the sliding detection model recognizes and processes the foot cup in the image to be tested, and outputs the position information of the recognized foot cup in the image to be tested, and then obtains the position information output by the sliding detection model to facilitate subsequent sliding detection of the material cage to be tested.

[0076] Please refer to Figure 8 In some embodiments, step 108 includes the following steps:

[0077] Step 802: If the foot cup coordinates are within the preset coordinate range, then there is no relative sliding between the material cage to be measured and the forks of the forklift.

[0078] Step 804: If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

[0079] Specifically, the foot cup coordinates refer to the central horizontal coordinates of the cage foot cup. By comparing the foot cup coordinates with a pre-set coordinate range, it is determined whether relative sliding has occurred between the cage to be tested and the forklift's forks. If the foot cup coordinates are within the preset coordinate range, it means that there has been no relative sliding between the cage to be tested and the forklift's forks; and if the foot cup coordinates are outside the coordinate range, it means that relative sliding has occurred between the cage to be tested and the forklift's forks. In this case, if the forklift is unmanned, the forklift needs to stop suddenly, place the cage to be tested in a temporary storage location, and then re-take the cage to be tested; if the forklift is manned, the forklift triggers an audible and visual alarm to remind nearby personnel to avoid it in time.

[0080] In some embodiments, as Figure 9 As shown, the cage sliding detection method includes the following steps:

[0081] Step 902 , obtaining a real-time video stream of the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup.

[0082] In step 902, two cameras can be installed at the left and right ends of the forklift's fork arm. Each camera is connected to the forklift's industrial computer network, which stores each camera's IP address and port number. The cameras can capture the two foot cups of each cage to be tested, monitoring whether the foot cups and the forklift are sliding relative to each other. The real-time video stream captured by the cameras is transmitted via the network to the forklift's industrial computer.

[0083] Step 904: Obtain the image to be tested corresponding to the cage to be tested according to the real-time video stream.

[0084] In step 904, the real-time video stream obtained in the above steps is extracted and processed to obtain an image to be tested corresponding to the cage to be tested.

[0085] It should be noted that the cage sliding detection method of the embodiment of the present application is a continuous detection, and is not targeted at a specific frame of image. Therefore, when the real-time video stream is extracted and processed to obtain the image to be tested corresponding to the cage to be tested, the image to be tested is any frame of image in the real-time video stream.

[0086] Step 906 : Detect the image to be detected according to the preset sliding detection model to obtain the coordinates of the foot cup in the cage image.

[0087] In step 906, the sliding detection model is trained by using the labeled original image data as input to a preset original detection algorithm model. The original detection algorithm model can adopt the YOLOV5 algorithm model. The specific process is as follows:

[0088] First, a large amount of collected original image data is annotated using an image annotation tool (such as labelImg) to mark out the foot cup in the original image data.

[0089] Then the original image data and the original image data after the foot cup is annotated are input into the YOLOV5 model for training, and the training generates a .pt (the format of the model trained by yolov5 based on pytorch.pt) model, that is, a sliding detection model is obtained.

[0090] The sliding detection model recognizes and processes the input image to be tested, identifies the position information of the cup in the cage image (i.e., the cup coordinates), and outputs the cup coordinates of the cup in the image to be tested. The cup coordinates refer to the central horizontal coordinates of the cup in the cage to be tested.

[0091] Step 908: determine whether the foot cup coordinates are within a preset coordinate range, and obtain the sliding detection result of the material cage to be tested.

[0092] In step 908, if the foot cup coordinates are within the preset coordinate range, it means that no relative sliding occurs between the material cage to be tested and the forklift; if the foot cup coordinates are outside the preset coordinate range, it means that relative sliding occurs between the material cage to be tested and the forklift.

[0093] Step 910: If the forklift is an unmanned forklift and the sliding detection result is that relative sliding occurs between the material cage to be tested and the forklift, a control signal is output to stop the unmanned forklift and place the material cage to be tested in a temporary storage location, and a warning message is output; if the forklift is a manned forklift and the sliding detection result is that relative sliding occurs between the material cage to be tested and the forklift, a warning message is output.

[0094] In step 910, if the forklift is unmanned and the sliding detection result indicates relative sliding between the cage to be tested and the forklift, a warning message and a control signal are output to cause the unmanned forklift to stop according to the control signal and place the cage to be tested in a temporary storage location. The forklift is then controlled to re-pick up the cage to be tested, thereby improving the safety of the forklift operation. Based on the warning message, the forklift triggers an audible and visual alarm to alert nearby personnel to avoid the forklift in a timely manner. If the forklift is manned and the sliding detection result indicates relative sliding between the cage to be tested and the forklift, a warning message is output to cause the forklift to trigger an audible and visual alarm to alert nearby personnel to avoid the forklift in a timely manner.

[0095] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0096] Based on the same inventive concept, the present application also provides a cage slide detection device for implementing the cage slide detection method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more cage slide detection device embodiments provided below can be found in the above-mentioned limitations of the cage slide detection method and will not be repeated here.

[0097] In some embodiments, as Figure 10As shown, a cage sliding detection device is provided, including: an acquisition module 1002, a processing module 1004, an identification and detection module 1006 and a sliding detection module 1008.

[0098] The acquisition module 1002 is used to acquire a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup.

[0099] The processing module 1004 is used to mark the cage image as the image to be tested, or to extract and process the cage video stream to obtain the image to be tested.

[0100] The identification and detection module 1006 is used to detect the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup.

[0101] The sliding detection module 1008 is used to determine whether the foot cup coordinates are within a preset coordinate range and obtain the sliding detection result of the material cage to be tested.

[0102] In some embodiments, the cage image or cage video stream is obtained by shooting the cage to be tested with a camera pre-installed on a forklift. The display interface corresponding to the cage image or cage video stream is provided with shooting auxiliary lines and an area of interest. The sliding detection device further includes a camera adjustment module:

[0103] The camera adjustment module is used to adjust the angle of the camera so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is in the area of interest.

[0104] In some embodiments, the region of interest includes a first region of interest and a second region of interest, the shooting auxiliary lines include a first auxiliary line, a second auxiliary line, and a third auxiliary line, the first auxiliary line intersects with the second auxiliary line and the third auxiliary line respectively, and the foot cup includes a first foot cup and a second foot cup.

[0105] In some embodiments, the first auxiliary line is perpendicular to the second auxiliary line and the third auxiliary line, and the third auxiliary line is parallel to the second auxiliary line.

[0106] The camera adjustment module includes a first camera adjustment unit and a second camera adjustment unit.

[0107] The first camera adjustment unit is used to adjust the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is in the first area of interest.

[0108] The second camera adjustment unit is used to adjust the roll angle of the camera so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup and the second foot cup is in the second area of interest.

[0109] In some embodiments, the recognition detection module 1006 includes an input unit and an acquisition unit.

[0110] The input unit is used to input the image to be tested into the trained sliding detection model, and the sliding detection model is used to perform recognition processing on the image to be tested to obtain the position information of the foot cup in the image to be tested.

[0111] The acquisition unit is used to obtain the position information output by the sliding detection model.

[0112] In some embodiments, the slide detection module 1008 includes a first detection unit and a second detection unit.

[0113] The first detection unit is used to determine that if the foot cup coordinates are within a preset coordinate range, there is no relative sliding between the material cage to be tested and the forks of the forklift.

[0114] The second detection unit is used to detect relative sliding between the material cage to be tested and the fork of the forklift if the foot cup coordinates are outside the preset coordinate range.

[0115] Each module in the above-mentioned cage slippage detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0116] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The communication interface, the display screen and the input device of the computer device are connected to the system bus via an I / O port. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for detecting the sliding of a material cage is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0117] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0119] Step 102, obtaining a cage image or a cage video stream obtained by photographing the cage to be tested; wherein the cage to be tested is placed on a forklift, and the cage to be tested includes a foot cup;

[0120] Step 104: Mark the cage image as the image to be tested, or extract and process the cage video stream to obtain the image to be tested;

[0121] Step 106: Detect the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup;

[0122] Step 108 , determining whether the foot cup coordinates are within a preset coordinate range, and obtaining a sliding detection result of the material cage to be tested.

[0123] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0124] Adjust the camera angle so that the shooting guide line passes through the center of the foot cup and the foot cup is within the area of interest.

[0125] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0126] Step 402 : Adjust the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is in the first region of interest.

[0127] Step 404 : Adjust the roll angle of the camera so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

[0128] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0129] Step 702: Input the image to be tested into the trained slide detection model. The slide detection model is used to perform recognition processing on the image to be tested to obtain the position information of the foot cup in the image to be tested.

[0130] Step 704: Obtain the position information output by the sliding detection model.

[0131] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0132] Step 802: If the foot cup coordinates are within the preset coordinate range, then there is no relative sliding between the material cage to be tested and the forks of the forklift;

[0133] Step 804: If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0135] Step 102, obtaining a cage image or a cage video stream obtained by photographing the cage to be tested; wherein the cage to be tested is placed on a forklift, and the cage to be tested includes a foot cup;

[0136] Step 104: Mark the cage image as the image to be tested, or extract and process the cage video stream to obtain the image to be tested;

[0137] Step 106: Detect the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup;

[0138] Step 108 , determining whether the foot cup coordinates are within a preset coordinate range, and obtaining a sliding detection result of the material cage to be tested.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0140] Adjust the camera angle so that the shooting guide line passes through the center of the foot cup and the foot cup is within the area of interest.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0142] Step 402 : Adjust the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is in the first region of interest.

[0143] Step 404 : Adjust the roll angle of the camera so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

[0144] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0145] Step 702: Input the image to be tested into the trained slide detection model. The slide detection model is used to perform recognition processing on the image to be tested to obtain the position information of the foot cup in the image to be tested.

[0146] Step 704: Obtain the position information output by the sliding detection model.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0148] Step 802: If the foot cup coordinates are within the preset coordinate range, then there is no relative sliding between the material cage to be tested and the forks of the forklift;

[0149] Step 804: If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

[0150] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0151] Step 102, obtaining a cage image or a cage video stream obtained by photographing the cage to be tested; wherein the cage to be tested is placed on a forklift, and the cage to be tested includes a foot cup;

[0152] Step 104: Mark the cage image as the image to be tested, or extract and process the cage video stream to obtain the image to be tested;

[0153] Step 106: Detect the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup;

[0154] Step 108 , determining whether the foot cup coordinates are within a preset coordinate range, and obtaining a sliding detection result of the material cage to be tested.

[0155] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0156] Adjust the camera angle so that the shooting guide line passes through the center of the foot cup and the foot cup is within the area of interest.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0158] Step 402 : Adjust the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is in the first region of interest.

[0159] Step 404 : Adjust the roll angle of the camera so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0161] Step 702: Input the image to be tested into the trained slide detection model. The slide detection model is used to perform recognition processing on the image to be tested to obtain the position information of the foot cup in the image to be tested.

[0162] Step 704: Obtain the position information output by the sliding detection model.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0164] Step 802: If the foot cup coordinates are within the preset coordinate range, then there is no relative sliding between the material cage to be tested and the forks of the forklift;

[0165] Step 804: If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

[0166] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0167] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A cage sliding detection method, characterized in that: The method comprises: Acquire a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup; Marking the cage image as the image to be tested, or extracting and processing the cage video stream to obtain the image to be tested; Detecting the image to be tested according to a preset sliding detection model to obtain position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup; It is determined whether the foot cup coordinates are within a preset coordinate interval to obtain a sliding detection result of the material cage to be tested.

2. The method according to claim 1, characterized in that The cage image or the cage video stream is obtained by photographing the cage to be tested with a camera pre-installed on a forklift, and a display interface corresponding to the cage image or the cage video stream is provided with shooting auxiliary lines and an area of interest; Before obtaining the cage image or cage video stream obtained by photographing the cage to be tested, the method further includes: The angle of the camera is adjusted so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is located in the region of interest.

3. The method according to claim 2, characterized in that The region of interest includes a first region of interest and a second region of interest, the shooting auxiliary lines include a first auxiliary line, a second auxiliary line and a third auxiliary line, the first auxiliary line intersects with the second auxiliary line and the third auxiliary line respectively, and the material cage to be tested includes two foot cups, which are named as a first foot cup and a second foot cup respectively; The adjusting the angle of the camera so that the shooting auxiliary line passes through the center of the foot cup and the foot cup is in the region of interest includes: Adjusting the yaw angle and pitch angle of the camera so that the first auxiliary line and the second auxiliary line intersect at the center of the first foot cup, and the first foot cup is within the first region of interest; The roll angle of the camera is adjusted so that the first auxiliary line and the third auxiliary line intersect at the center of the second foot cup, and the second foot cup is in the second region of interest.

4. The method according to claim 1, wherein The detecting the image to be tested according to a preset sliding detection model to obtain the position information of the foot cup in the image to be tested includes: Inputting the image to be tested into a trained sliding detection model, wherein the sliding detection model is used to perform recognition processing on the image to be tested to obtain position information of the foot cup in the image to be tested; The position information output by the sliding detection model is obtained.

5. The method according to any one of claims 1 to 4, characterized in that The sliding detection model is obtained by training the labeled original image data as the input of a preset original detection algorithm model.

6. The method according to any one of claims 1 to 4, characterized in that The determining whether the foot cup coordinates are within a preset coordinate interval to obtain the sliding detection result of the material cage to be tested includes: If the foot cup coordinates are within the preset coordinate interval, no relative sliding occurs between the material cage to be tested and the fork of the forklift; If the foot cup coordinates are outside the preset coordinate range, relative sliding occurs between the material cage to be measured and the forks of the forklift.

7. A cage sliding detection device, characterized in that: The device comprises: An acquisition module is used to acquire a material cage image or a material cage video stream obtained by photographing the material cage to be tested; wherein the material cage to be tested is placed on the forks of a forklift, and the material cage to be tested includes a foot cup; A processing module, configured to mark the cage image as an image to be tested, or extract and process the cage video stream to obtain the image to be tested; an identification and detection module, configured to detect the image to be tested according to a preset sliding detection model to obtain position information of the foot cup in the image to be tested; wherein the position information includes the coordinates of the foot cup; The sliding detection module is used to determine whether the foot cup coordinates are within a preset coordinate range and obtain the sliding detection result of the material cage to be tested.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images