Counting identification method and system, readable storage medium and computer program product

Video information is obtained through the camera and trajectory prediction and object detection algorithms are used to automatically identify the motion trajectory and wood quantity of lifting equipment, solving the problems of counting error and long periods during manual statistics, and achieving fast and accurate wood staples.

CN118942007APending Publication Date: 2024-11-12FUJIAN ZHONGKE TIANXUN INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410996402.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When manual statistics on wood, counting errors are prone to occur, resulting in a long collection cycle.

Method used

The video information of the lifting equipment is obtained through the camera, and the trajectory prediction model is used to predict the motion trajectory and the target shooting position. Combined with the target tracking algorithm and detection algorithm, the number of targets to be transferred is automatically identified and the accurate counting results are generated.

Benefits of technology

The number of targets to be transported is quickly and accurately identified, which significantly shortens the cargo cycle and reduces manual counting errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942007A_ABST
    Figure CN118942007A_ABST
Patent Text Reader

Abstract

The invention discloses a counting recognition method and system, a readable storage medium and a computer program product, relates to the technical field of artificial intelligence, and solves the problem that the wood tallying period is relatively long in a manual tallying mode at present. The method comprises the steps that when it is determined that a transfer device of the hoisting equipment executes transfer operation, video information is continuously obtained through a camera, motion trail prediction is conducted through a trail prediction model, and predicted motion trail information and a target shooting position of the transfer device are determined; extracting current actual movement track information of the transfer device by using a target tracking algorithm at preset time intervals, when detecting that a target position matched with the target shooting position exists in the current actual movement track information, intercepting an image frame corresponding to the image acquisition time from the current video information, and amplifying the image frame to obtain a target position matched with the target shooting position; and carrying out quantity identification on the amplified to-be-transferred targets by utilizing a target detection algorithm to obtain a target quantity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a counting and recognition method, system, readable storage medium and computer program product. Background Art

[0002] As an important cargo distribution center, ports need to handle a large amount of import and export goods every day. Take wood as an example. After the wood is transported to the port, it needs to be lifted from the cabin by the quay crane arm and then placed on the transfer truck. At present, wood tallying mainly relies on manual counting. However, manual tallying is easily affected by factors such as fatigue and lack of concentration, which can easily lead to counting errors. When counting errors occur, rechecking is required, which makes the wood tallying cycle longer. Summary of the invention

[0003] The embodiments of the present application provide a counting and identification method, system, readable storage medium and computer program product, the main purpose of which is to solve the problem that the current manual tallying method results in a long timber tallying cycle.

[0004] According to the first aspect of the present application, a counting and recognition method is provided, comprising:

[0005] When determining that a transfer device of the lifting equipment performs a transfer operation, continuously acquiring video information including a motion state of the transfer device through a camera, wherein a plurality of objects to be transferred are arranged in the transfer device;

[0006] Determine target video information for predicting the trajectory, and based on the target video information, use a trajectory prediction model to predict the motion trajectory, determine the predicted motion trajectory information of the transfer device, and obtain the target shooting position in the predicted motion trajectory information;

[0007] Extracting the current actual motion trajectory information of the transfer device from the continuously acquired video information by using a target tracking algorithm at preset time intervals, and when detecting that the current actual motion trajectory information has a target position matching the target shooting position, determining the time corresponding to the target position as the image acquisition time, and intercepting the image frame corresponding to the image acquisition time from the continuously acquired video information;

[0008] The multiple targets to be transferred in the image frame are enlarged to obtain a target image frame including multiple enlarged targets to be transferred, and the target detection algorithm is used to identify the number of the multiple enlarged targets to be transferred in the target image frame to obtain the target number of the multiple targets to be transferred.

[0009] Optionally, based on the target video information, using a trajectory prediction model to predict the motion trajectory, determining the predicted motion trajectory information of the transfer device, and obtaining the target shooting position in the predicted motion trajectory information, further includes:

[0010] Extracting the initial actual motion trajectory information of the transfer device from the target video information using the target tracking algorithm;

[0011] The initial actual motion trajectory information is input into the trajectory prediction model to obtain the predicted motion trajectory information of the transfer device output by the trajectory prediction model and the target shooting position in the predicted motion trajectory information.

[0012] Optionally, based on the target video information, using a trajectory prediction model to predict the motion trajectory, determining the predicted motion trajectory information of the transfer device, and obtaining the target shooting position in the predicted motion trajectory information, further includes:

[0013] Acquire a trajectory training data set, wherein the trajectory training data set includes incomplete motion trajectory information and at least one complete motion trajectory information corresponding to each incomplete motion trajectory information, and a target position in each complete motion trajectory information is marked with label information for indicating the target shooting position, wherein the incomplete motion trajectory information is the motion trajectory information of the transfer device between the start time of the transfer device starting to rotate and the preset time, and the trajectory length corresponding to the complete motion trajectory information is greater than the trajectory length of the incomplete motion trajectory information;

[0014] A long short-term memory network is obtained, and the long short-term memory network is trained using the trajectory training data set, and model parameters are updated to obtain the trajectory prediction model.

[0015] Optionally, when detecting that the current motion trajectory information includes a target position matching the target shooting position, determining that the time corresponding to the target position is before the image acquisition time further includes:

[0016] Extracting at least one to-be-determined position from the current motion trajectory information, and acquiring a first coordinate of each to-be-determined position in the at least one to-be-determined position;

[0017] A second coordinate of the target shooting position is determined, and based on the first coordinate and the second coordinate, it is determined whether there is a target position matching the target shooting position in the at least one position to be determined.

[0018] Optionally, after intercepting the image frame corresponding to the image acquisition time from the continuously acquired video information, the method further includes:

[0019] Performing denoising processing on the image frame to obtain a denoised image frame, and performing image enhancement processing on the denoised image frame to obtain an enhanced image frame;

[0020] The enhanced image frame is subjected to distortion elimination processing to obtain an image frame after distortion elimination, and the image frame after distortion elimination is subjected to stretching processing to obtain a processed image frame.

[0021] Optionally, the method of using a target detection algorithm to identify the number of the multiple magnified targets to be transported in the target image frame and obtaining the number of the multiple targets to be transported includes:

[0022] The actual number of the multiple targets to be transferred set by the transfer device is obtained. If the actual number is inconsistent with the target number, a warning message is generated to indicate that the identification result is inaccurate, and the warning message is sent to the terminal held by the manager, so that the manager can troubleshoot the problem based on the warning message when receiving the warning message at the terminal.

[0023] According to a second aspect of the present application, there is provided a counting and recognition system, comprising: a camera, a detection unit, a control unit and a processing unit;

[0024] The control unit is connected to the camera, the detection unit and the processing unit;

[0025] The camera is arranged at a preset position in the area where the lifting equipment is located, and is used to continuously collect video information including the lifting equipment when receiving a video collection instruction issued by the control unit, and send the continuously collected video information to the detection unit;

[0026] The detection unit is used to generate a detection signal for indicating that the transfer device of the lifting equipment starts to move when detecting that the transfer device of the lifting equipment starts to move in the continuously collected video information, and send the detection signal to the control unit;

[0027] The control unit is used to generate a counting identification instruction upon receiving the detection signal, and send the counting identification instruction to the processing unit;

[0028] The processing unit is used to respond to the counting and recognition instruction and execute the counting and recognition method according to any one of the above-mentioned first aspects.

[0029] Optionally, the counting and identification system further comprises: an early warning unit;

[0030] The early warning unit is used to determine the target number of multiple targets to be transferred obtained according to the counting and identification method described in any one of the first aspects above, and to obtain the actual number of the multiple targets to be transferred set by the transfer device of the lifting equipment. If the actual number is inconsistent with the target number, an early warning message is generated to indicate that the identification result is inaccurate, and the early warning message is sent to the terminal held by the manager, so that the manager can troubleshoot the problem based on the early warning message when the terminal receives the early warning message.

[0031] According to a third aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0032] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0033] By means of the above technical scheme, the present application provides a counting and recognition method, system, readable storage medium and computer program product. The present application determines the target video information used to predict the trajectory, and uses the trajectory prediction model to predict the motion trajectory of the transfer device of the lifting equipment, thereby obtaining the predicted motion trajectory information of the transfer device and the best shooting position in the predicted motion trajectory information, that is, the target shooting position. Then, the target tracking algorithm is used to extract the current actual motion trajectory information of the transfer device from the continuously acquired video information at preset time intervals. When a target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is determined as the image acquisition time. An image frame corresponding to the image acquisition time is captured from the continuously acquired video information, and then a plurality of targets to be transferred in the image frame are enlarged to obtain a target image frame including a plurality of enlarged targets to be transferred, and a target detection algorithm is used to identify the quantity of the plurality of enlarged targets to be transferred in the target image frame, so as to obtain the target quantity of the plurality of targets to be transferred; by establishing a relationship between the operating area of ​​the lifting equipment and the end faces of the plurality of targets to be transferred, a control method adapted to different states during the operation process is adopted, and complex and difficult operations are transformed into inertial trajectory following and waiting, and the quantity of the plurality of targets to be transferred is automatically identified, that is, the quantity of the plurality of targets to be transferred can be identified quickly and accurately, thereby greatly shortening the tallying cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1A A flowchart of a counting and recognition method provided by an embodiment of the present application is shown;

[0035] Figure 1BA schematic diagram showing the structure of a target detection algorithm of a counting and recognition method provided in an embodiment of the present application is shown;

[0036] Figure 2 A flowchart of another counting and recognition method provided by an embodiment of the present application is shown;

[0037] Figure 3A A schematic diagram of the structure of a counting and recognition system provided in an embodiment of the present application is shown;

[0038] Figure 3B A structural schematic diagram of a counting and recognition system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0040] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0042] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0043] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0044] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0045] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0046] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0047] The present application embodiment provides a counting and recognition method, such as Figure 1A As shown, including:

[0048] 101. When determining that a transfer device of the lifting equipment performs a transfer operation, continuously obtain video information including a movement state of the transfer device through a camera.

[0049] In an embodiment of the present application, when the transfer device of the lifting equipment starts to move, that is, when the transfer operation starts, the camera will continue to collect video information of the movement state of the transfer device. This video information is composed of an image sequence, and each image frame corresponds to a collection time. It should be noted that the camera not only collects video when the transfer device performs the transfer operation, but also monitors the target area when the transfer device does not perform the transfer operation. It should also be noted that due to the large size of the transfer device, such as a grab machine, its movement range is also large, so the camera is set at a distance from the transfer device. At the same time, a plurality of targets to be transferred are arranged in the transfer device. The transfer device can be a device that can realize the transfer of objects, such as the boom bucket of a grab machine. The target to be transferred can be a cylindrical, rectangular, or long strip target, such as wood, steel coils, plates, etc.

[0050] It should be noted that this transfer operation can transfer the target to be transferred from the cabin to the truck, or from the truck to the cabin. In addition to the above scenarios, it is also applicable to transferring the target to be transferred from one place to another on land, or transferring the target to be transferred from one ship to another on the water or sea.

[0051] 102. Determine target video information for predicting the trajectory, and based on the target video information, use the trajectory prediction model to predict the motion trajectory, determine the predicted motion trajectory information of the transfer device, and obtain the target shooting position in the predicted motion trajectory information;

[0052] In the embodiment of the present application, the trajectory prediction algorithm is an optimal algorithm that has been trained in advance and can be used directly here. The target video information used to predict the trajectory here can be selected according to time. For example, the time corresponding to the transfer device of the lifting equipment starting to perform the transfer operation can be used as the initial time. A target time interval can be predetermined, and the time obtained by adding the initial time to the target time interval is used as the selection time, and the video information corresponding to the selection time is used as the target video information. After determining the target video information used to predict the trajectory, the motion trajectory is predicted based on the target video information using the trajectory prediction model, the predicted motion trajectory information of the transfer device can be determined, and the target shooting position in the predicted motion trajectory information can be obtained. It should be noted that the target shooting position can be understood as the position where the end faces of multiple targets to be transferred are facing the camera.

[0053] 103. At preset time intervals, the current actual motion trajectory information of the transfer device is extracted from the continuously acquired video information using a target tracking algorithm. When a target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is determined as the image acquisition time, and an image frame corresponding to the image acquisition time is captured from the continuously acquired video information.

[0054] In an embodiment of the present application, after obtaining the target shooting position in the predicted motion trajectory information, the current actual motion trajectory information of the transfer device is extracted from the continuously acquired video information using a target tracking algorithm at preset time intervals. For example, every 1 second, when a target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is determined as the image acquisition time, and an image frame corresponding to the image acquisition time is captured from the continuously acquired video information.

[0055] It should be noted that the target tracking algorithm preferably uses the kernel correlation filter tracking algorithm (KCF). For example, in a port scenario, wood is identified and the kernel correlation filter algorithm can be used to ensure that the wood is continuously tracked in the entire image sequence. The kernel correlation filter tracking algorithm is a prior art and can be easily obtained by those skilled in the art, so it will not be described in detail here.

[0056] 104. A plurality of targets to be transferred in the image frame are enlarged to obtain a target image frame including a plurality of enlarged targets to be transferred, and a target detection algorithm is used to identify the number of the plurality of enlarged targets to be transferred in the target image frame to obtain the number of the plurality of targets to be transferred.

[0057] In an embodiment of the present application, after capturing an image frame corresponding to the image acquisition time from the continuously acquired video information, the multiple targets to be transferred in the image frame are enlarged. Lossless enlargement technology can be used here to enlarge the multiple targets to be transferred in the image frame. Lossless enlargement technology is a prior art, which can supplement the noise and jaggedness of the enlarged image to achieve lossless enlargement of the image. Afterwards, the target detection algorithm is used to identify the number of multiple enlarged targets to be transferred in the target image frame to obtain the target number of the multiple targets to be transferred.

[0058] It should be noted that the YOLO algorithm is mainly used in the prior art to detect targets in images. However, in the port scene, due to the complexity of the port environment, it is easily affected by the natural environment of weather and lighting. In addition, when identifying the quantity of wood, the size, type and shape of the wood will also affect the image quality. In other words, the small target problem, image shadow and overexposure problems are huge challenges to the detection ability of the model. In order to adapt to the above problems, the target detection algorithm in this application is an improvement on the YOLO algorithm, such as Figure 1B As shown in the figure, the P2 detection layer, P3 detection layer, P4 detection layer and P5 detection layer are added. The added P2 detection layer, P3 detection layer, P4 detection layer and P5 detection layer make the target detection algorithm pay more attention to the pixel information of small targets, so that the target detection algorithm can detect and locate small targets more sensitively, and enhance the detection accuracy of small targets. At the same time, the Inner-IoU loss function is introduced in the target detection algorithm, and the Inner-CIoU loss function is proposed in combination with the model's own loss function CIoU. Compared with the traditional loss function, the Inner-IoU loss function not only considers the overall overlap of the two bounding boxes when calculating IoU, but also introduces auxiliary bounding boxes to further refine the evaluation of the overlapping area. This method can improve the discrimination in the case of high overlap, thereby accelerating the convergence process of the model and improving the detection performance. The Inner-CIoU loss function can provide higher discrimination than the original CIoU, which helps the model to further optimize the position and size of the predicted bounding box. By introducing auxiliary bounding boxes and the Inner-IoU loss function, the convergence process of the model can be accelerated and the training efficiency can be improved. The results show that using the Inner-CIoU loss function can significantly improve the model's performance for wood detection. The Inner-CIoU loss function formula is as follows:

[0059]

[0060] inter=(min(b r gt , b r )-max(b l gt , bl ))*(min(b b gt , b b )-max(b t gt , b t )),

[0061] union=(W gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter,

[0062]

[0063] L inner-CIoU =L CIoU +IOU-IoU inner ,

[0064] Among them, the GT box and the Anchor box are represented as b gt and b, the center point of the GT box is It means that (x c ,y c ) represents the center point of the Anchor box, and the width and height of the GT box are w gt and h gt Indicates, and the width and height of the Anchor box are represented by w and h, the variable ratio represents the corresponding scaling factor, the subscript l represents the left border of the border, the subscript r represents the right border of the border, the subscript t represents the upper border of the border, the subscript b represents the lower border of the border, inter represents the intersection of the two box areas, union represents the union of the two box areas, IoU represents the intersection-over-union ratio, L CIoU Represents the CIoU loss function, which is another way to calculate the intersection-over-union ratio. inner-CIoU Represents the inner-CIou loss function in this application.

[0065] Furthermore, when the model is deployed, it is necessary to first perform output separation processing on the four detection heads of the model to improve the detection speed of the model. In order to improve the detection effect, the annotation method and model parameters including confidence threshold and IOU threshold are adjusted accordingly. That is, by refining the category type and annotation size, the model can give a variety of output states and eliminate the category confusion problem to a certain extent, so that the algorithm can effectively coordinate the camera movement, zooming and snapshot timing according to the model output. According to the overlap degree of the category and the size of the detection frame, it can be inferred whether the camera position is correct, whether the grab angle is suitable for camera zooming, and whether the snapshot timing is correct; for the model, different confidence thresholds indicate different numbers of filtered prediction frames, and different IOU thresholds indicate whether the misidentified objects can be filtered out when the prediction results are processed by non-maximum suppression (NMS). The factors affecting the setting of these two parameters are mainly the detection effect and confusion degree of the model for each category. Through the statistical and inductive prediction of all data, the detection score range and the false detection result range of each category are analyzed, and the appropriate confidence threshold and IOU threshold are selected. The model performs the first pass screening of all detection results of the input image through the confidence threshold, and then performs the second screening of the detection results through the NMS function, and then gives the final prediction result.

[0066] The method provided by the embodiment of the present application determines the target video information for predicting the trajectory, and predicts the motion trajectory of the transfer device of the lifting equipment by using the trajectory prediction model, so as to obtain the predicted motion trajectory information of the transfer device and the best shooting position in the predicted motion trajectory information, that is, the target shooting position. Then, the current actual motion trajectory information of the transfer device is extracted from the continuously acquired video information by using the target tracking algorithm at every preset time interval. When the target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is determined as the image acquisition time, and the image frame corresponding to the image acquisition time is intercepted from the continuously acquired video information. Then, the multiple targets to be transferred in the image frame are enlarged to obtain a target image frame including multiple enlarged targets to be transferred, and the target detection algorithm is used to identify the number of the multiple enlarged targets to be transferred in the target image frame, so as to obtain the target number of the multiple targets to be transferred. By establishing the relationship between the operation area of ​​the lifting equipment and the end faces of the multiple targets to be transferred, the control mode of different states in the operation process is adapted, and the complex and difficult operation is transformed into inertial trajectory following and waiting, and the number of the multiple targets to be transferred is automatically identified, that is, the number of the multiple targets to be transferred can be quickly and accurately identified, which greatly shortens the tally cycle.

[0067] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation of this embodiment, the embodiment of the present application provides another counting and recognition method, such as Figure 2As shown, including:

[0068] 201. When determining that the transfer device of the lifting equipment performs a transfer operation, the video information including the movement status of the transfer device is continuously obtained through the camera.

[0069] In the embodiment of the present application, the execution subject may be a system that executes the counting and recognition method. When the system determines that the transfer device of the lifting equipment performs a transfer operation, the system continuously obtains video information including the motion state of the transfer device through a camera. For example, in a port scene, the grab arm is used to lift wood from the cabin and then put it on the transfer truck. When it is determined that the grab arm performs the operation of transferring wood, the video information is continuously obtained through the camera, and the video information contains the motion state of the arm.

[0070] 202. Based on the target video information, a trajectory prediction model is used to predict the motion trajectory, determine the predicted motion trajectory information of the transfer device, and obtain the target shooting position in the predicted motion trajectory information.

[0071] In an embodiment of the present application, a target tracking algorithm is first used to extract the initial actual motion trajectory information of the transfer device from the target video information; then the initial actual motion trajectory information is input into a trajectory prediction model to obtain the predicted motion trajectory information of the transfer device output by the trajectory prediction model and the target shooting position in the predicted motion trajectory information.

[0072] It should be noted that the trajectory prediction model is an optimal algorithm trained in advance, that is, first, a trajectory training data set is obtained, where the trajectory training data set includes incomplete motion trajectory information and at least one complete motion trajectory information corresponding to each incomplete motion trajectory information, and a target position in each complete motion trajectory information is marked with label information for indicating the target shooting position, wherein the incomplete motion trajectory information is the motion trajectory information of the transfer device between the start time of the transfer device starting to rotate and the preset time, and the trajectory length corresponding to the complete motion trajectory information is greater than the trajectory length of the incomplete motion trajectory information; then a long short-term memory network is obtained, the long short-term memory network is trained using the trajectory training data set, the model parameters are updated, and the trajectory prediction model is obtained.

[0073] It should be noted that it takes some time from the start of the transfer of the target to the transfer to the target location. A preset time can be determined, and the motion trajectory information corresponding to the start of the transfer time to the preset time can be used as incomplete motion trajectory information. Then, the motion trajectory corresponding to the start of the transfer time to the transfer completion time is obtained as the complete motion trajectory information. Since these contents are obtained in advance, there will be an optimal shooting position on a complete motion trajectory, and this optimal shooting position will be marked.

[0074] It should be further explained that LSTM is a deep learning model suitable for processing sequence data. Its internal structure can effectively memorize and utilize long-term dependencies in the input sequence, which helps to predict future trajectories more accurately. LSTM can learn and represent complex nonlinear relationships. Compared with traditional linear models, it is more suitable for trajectory prediction problems and can better capture patterns and regularities in trajectory data. Traditional methods usually require fixed-length input sequences, while LSTM can process variable-length sequence data and is more suitable for tasks such as trajectory prediction that require consideration of historical data of different lengths. Moreover, as an end-to-end learning method, LSTM can learn feature representations directly from data without manually designing feature extractors, making it more adaptable to various types of trajectory data. Therefore, the trajectory prediction method based on LSTM has better flexibility, accuracy and generalization ability when processing trajectory data, and is more suitable for complex trajectory prediction tasks than traditional methods.

[0075] 203. At preset time intervals, use a target tracking algorithm to extract current actual motion trajectory information of the transfer device from the continuously acquired video information.

[0076] 204. Determine whether there is a target position matching the target shooting position in the at least one to-be-determined position.

[0077] In the embodiment of the present application, at least one position to be determined can be extracted from the current motion trajectory information, and the first coordinates of each position to be determined in the at least one position to be determined are obtained, for example, (x1, y1), (x2, y2) and (x3, y3). The second coordinates of the target shooting position are determined in advance, for example, (x0, y0). For each first coordinate, x is calculated. i The square of the first difference from x0 and the calculation of y i The square of the second difference with y0 is calculated, and the square of the sum of the square of the first difference and the square of the second difference is taken as the square root to obtain the target error. Then, it is determined which target error among the target errors corresponding to the multiple first coordinates is smaller than the preset error, and the position corresponding to the target error being smaller than the preset error is taken as the target position.

[0078] 205. When it is detected that the current actual motion trajectory information has a target position matching the target shooting position, determine the time corresponding to the target position as the image acquisition time, and capture an image frame corresponding to the image acquisition time from the continuously acquired video information.

[0079] In an embodiment of the present application, since the motion trajectory information includes not only multiple positions and the coordinates corresponding to each position, but also the time corresponding to each position, when a target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is used as the image acquisition time. At this time, the image frame corresponding to the image acquisition time can be captured from the continuously acquired video information.

[0080] In an optional implementation of an embodiment of the present application, after capturing an image frame, the image frame may be preprocessed, that is, the image frame is first denoised to obtain a denoised image frame, the denoised image frame is then enhanced to obtain an enhanced image frame, the enhanced image frame is then subjected to distortion elimination processing to obtain a distortion eliminated image frame, and finally the distortion eliminated image frame is stretched to obtain a processed image frame.

[0081] 206. A plurality of objects to be transferred in the image frame are enlarged to obtain an object image frame including the plurality of enlarged objects to be transferred.

[0082] 207. Utilize a target detection algorithm to identify the number of multiple magnified targets to be transferred in the target image frame, and obtain the number of the multiple targets to be transferred.

[0083] 208. Check the recognition result.

[0084] In the embodiment of the present application, the actual number of multiple to-be-transferred targets set by the transfer device can be obtained first. If the actual number is inconsistent with the target number, a warning message indicating that the identification result is inaccurate is generated, and the warning message is sent to the terminal held by the manager, so that the manager can perform problem troubleshooting based on the warning message when the terminal receives the warning message. Inspection can be performed at regular intervals, so that problems can be discovered in time and troubleshooted.

[0085] In an optional implementation of the embodiment of the present application, the identification result can be tested after all the targets to be transferred have been transferred. For example, all the wood in the cabin is transferred to the truck, and the bucket will be automatically counted during each transfer. For example, 5 transfers are required to complete the transfer. The first time, 10 woods, the second time, 11 woods, the third time, 10 woods, the fourth time, 10 woods, and the fifth time, 9 woods. The total number of transferred woods is 50. The number of woods transported in the cabin can be determined in advance. For example, the actual number is also 50, which means that there is no problem with the identification result. If the actual number is 49, it means that the current identification result is inaccurate and needs to be checked by relevant technical personnel.

[0086] The method provided by the embodiment of the present application determines the target video information for predicting the trajectory, and predicts the motion trajectory of the transfer device of the lifting equipment by using the trajectory prediction model, so as to obtain the predicted motion trajectory information of the transfer device and the best shooting position in the predicted motion trajectory information, that is, the target shooting position. Then, the current actual motion trajectory information of the transfer device is extracted from the continuously acquired video information by using the target tracking algorithm at every preset time interval. When the target position matching the target shooting position is detected in the current actual motion trajectory information, the time corresponding to the target position is determined as the image acquisition time, and the image frame corresponding to the image acquisition time is intercepted from the continuously acquired video information. Then, the multiple targets to be transferred in the image frame are enlarged to obtain a target image frame including multiple enlarged targets to be transferred, and the target detection algorithm is used to identify the number of the multiple enlarged targets to be transferred in the target image frame, so as to obtain the target number of the multiple targets to be transferred. By establishing the relationship between the operation area of ​​the lifting equipment and the end faces of the multiple targets to be transferred, the control mode of different states in the operation process is adapted, and the complex and difficult operation is transformed into inertial trajectory following and waiting, and the number of the multiple targets to be transferred is automatically identified, that is, the number of the multiple targets to be transferred can be quickly and accurately identified, which greatly shortens the tally cycle.

[0087] Further, as a specific implementation of the method described in FIG. 1, as shown in FIG. Figure 3A As shown, an embodiment of the present invention provides a counting and recognition system, including: a camera, a detection unit, a control unit and a processing unit.

[0088] Specifically, the control unit is connected to the camera, the detection unit and the processing unit; the camera is set at a preset position in the area where the lifting equipment is located, and is used to continuously collect video information including the lifting equipment when receiving a video acquisition instruction issued by the control unit, and send the continuously collected video information to the detection unit; the detection unit is used to generate a detection signal for indicating that the transfer device starts to move when it detects that the transfer device of the lifting equipment starts to move in the continuously collected video information, and send the detection signal to the control unit; the control unit is used to generate a counting recognition instruction when receiving the detection signal, and send the counting recognition instruction to the processing unit; the processing unit is used to respond to the counting recognition instruction and execute the counting recognition method.

[0089] Furthermore, if Figure 3BAs shown, the counting and identification system further includes an early warning unit. The early warning unit is used to determine the target number of the multiple objects to be transferred obtained according to the counting and identification method, and to obtain the actual number of the multiple objects to be transferred set by the transfer device of the lifting equipment. If the actual number is inconsistent with the target number, an early warning message indicating that the identification result is inaccurate is generated, and the early warning message is sent to the terminal held by the manager, so that the manager can perform problem troubleshooting based on the early warning message when the terminal receives the early warning message.

[0090] The system provided by the embodiment of the present application includes a camera, a detection unit, a control unit, a processing unit and an early warning unit; the control unit controls the camera to continuously obtain video information of the target area, and the video information includes the movement status of the transfer device of the lifting equipment. When the detection unit detects that the transfer device of the lifting equipment starts to move, the control unit controls the processing unit to count the wood based on the continuously obtained video information to obtain the target number of multiple targets to be transferred, and the early warning unit can issue an early warning based on the target number of multiple targets to be transferred; through the above process, the targets to be transferred are automatically counted and warned, which effectively improves the tallying efficiency and the tallying cycle.

[0091] In an exemplary embodiment, the present application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present application are implemented.

[0092] If the module / unit integrated in the computer unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0093] The present application also provides a computer program product, comprising a computer program, which implements the steps of the method described above when executed by a processor.

[0094] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0095] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A counting and recognition method, characterized in that: include: When determining that a transfer device of the lifting equipment performs a transfer operation, continuously acquiring video information including a motion state of the transfer device through a camera, wherein a plurality of objects to be transferred are arranged in the transfer device; Determine target video information for predicting the trajectory, and based on the target video information, use a trajectory prediction model to predict the motion trajectory, determine the predicted motion trajectory information of the transfer device, and obtain the target shooting position in the predicted motion trajectory information; Extracting the current actual motion trajectory information of the transfer device from the continuously acquired video information by using a target tracking algorithm at preset time intervals, and when detecting that the current actual motion trajectory information has a target position matching the target shooting position, determining the time corresponding to the target position as the image acquisition time, and intercepting the image frame corresponding to the image acquisition time from the continuously acquired video information; The multiple targets to be transferred in the image frame are enlarged to obtain a target image frame including multiple enlarged targets to be transferred, and the target detection algorithm is used to identify the number of the multiple enlarged targets to be transferred in the target image frame to obtain the target number of the multiple targets to be transferred.

2. The counting and recognition method according to claim 1, characterized in that: The method further comprises: performing motion trajectory prediction based on the target video information by using a trajectory prediction model to determine the predicted motion trajectory information of the transfer device, and obtaining the target shooting position in the predicted motion trajectory information. Extracting the initial actual motion trajectory information of the transfer device from the target video information using the target tracking algorithm; The initial actual motion trajectory information is input into the trajectory prediction model to obtain the predicted motion trajectory information of the transfer device output by the trajectory prediction model and the target shooting position in the predicted motion trajectory information.

3. The counting and recognition method according to claim 1, characterized in that: The method further comprises: performing motion trajectory prediction based on the target video information by using a trajectory prediction model to determine the predicted motion trajectory information of the transfer device, and obtaining the target shooting position in the predicted motion trajectory information. Acquire a trajectory training data set, wherein the trajectory training data set includes incomplete motion trajectory information and at least one complete motion trajectory information corresponding to each incomplete motion trajectory information, and a target position in each complete motion trajectory information is marked with label information for indicating the target shooting position, wherein the incomplete motion trajectory information is the motion trajectory information of the transfer device between the start time of the transfer device starting to rotate and the preset time, and the trajectory length corresponding to the complete motion trajectory information is greater than the trajectory length of the incomplete motion trajectory information; A long short-term memory network is obtained, and the long short-term memory network is trained using the trajectory training data set, and model parameters are updated to obtain the trajectory prediction model.

4. The counting and recognition method according to claim 1, characterized in that: When detecting that the current motion trajectory information includes a target position matching the target shooting position, determining that the time corresponding to the target position is before the image acquisition time, further comprising: Extracting at least one to-be-determined position from the current motion trajectory information, and acquiring a first coordinate of each to-be-determined position in the at least one to-be-determined position; A second coordinate of the target shooting position is determined, and based on the first coordinate and the second coordinate, it is determined whether there is a target position matching the target shooting position in the at least one position to be determined.

5. The counting and recognition method according to claim 1, characterized in that: After the image frame corresponding to the image acquisition time is intercepted from the continuously acquired video information, the method further includes: Performing denoising processing on the image frame to obtain a denoised image frame, and performing image enhancement processing on the denoised image frame to obtain an enhanced image frame; The enhanced image frame is subjected to distortion elimination processing to obtain an image frame after distortion elimination, and the image frame after distortion elimination is subjected to stretching processing to obtain a processed image frame.

6. The counting and recognition method according to claim 1, characterized in that: The method of using a target detection algorithm to identify the number of the plurality of magnified targets to be transported in the target image frame and obtaining the number of the plurality of targets to be transported includes: The actual number of the multiple targets to be transferred set by the transfer device is obtained. If the actual number is inconsistent with the target number, a warning message is generated to indicate that the identification result is inaccurate, and the warning message is sent to the terminal held by the manager, so that the manager can troubleshoot the problem based on the warning message when receiving the warning message at the terminal.

7. A counting and recognition system, characterized in that: include: Camera, detection unit, control unit and processing unit; The control unit is connected to the camera, the detection unit and the processing unit; The camera is arranged at a preset position in the area where the lifting equipment is located, and is used to continuously collect video information including the lifting equipment when receiving a video collection instruction issued by the control unit, and send the continuously collected video information to the detection unit; The detection unit is used to generate a detection signal for indicating that the transfer device of the lifting equipment starts to move when detecting that the transfer device of the lifting equipment starts to move in the continuously collected video information, and send the detection signal to the control unit; The control unit is used to generate a counting identification instruction upon receiving the detection signal, and send the counting identification instruction to the processing unit; The processing unit is used to respond to the counting and recognition instruction and execute the counting and recognition method according to any one of claims 1-6.

8. The counting and recognition system according to claim 1, characterized in that: Also includes: Early warning unit; The early warning unit is used to determine the target number of multiple targets to be transferred obtained according to the counting and identification method described in any one of claims 1 to 6, and to obtain the actual number of the multiple targets to be transferred set by the transfer device of the lifting equipment. If the actual number is inconsistent with the target number, an early warning message is generated to indicate that the identification result is inaccurate, and the early warning message is sent to the terminal held by the manager, so that the manager can troubleshoot the problem based on the early warning message when the terminal receives the early warning message.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.