Box key point detection method and device, computing device, and computer-readable storage medium

Through the cascading network model, the box image is detected using posture and face detection technology, which solves the problem of low accuracy and efficiency of box vertex detection in the logistics industry, and achieves higher accuracy key point positioning of box.

CN112101342BActive Publication Date: 2024-12-03SF TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910521575.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-06-17
Publication Date
2024-12-03
Estimated Expiration
2039-06-17

Smart Images

  • Figure CN112101342B_ABST
    Figure CN112101342B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention disclose a method and apparatus for detecting key points of a box, a computing device, and a computer-readable storage medium. The method comprises the following steps: obtaining first image data, wherein the first image data comprises data of an image of a first box; detecting the first image data using a first detection network model to output first detection result data, wherein the first detection result data comprises first coordinate data of a first key point of the first box; obtaining second image data of a predetermined size corresponding to the first key point from the first image data; detecting the second image data using a second detection network model to output second detection result data, wherein the second detection result data comprises at least second coordinate data of the first key point. The embodiments of the present invention can perform accurate intelligent detection of key points of a box.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of computer technology, and in particular to a box key point detection method and device, a computing device, and a computer-readable storage medium. [Background technology]

[0002] The logistics industry has a demand for accurate measurement of cargo volume.

[0003] At present, the volume of cargo is generally measured by manual measurement, which has low accuracy and low efficiency.

[0004] At present, there is no technology in the art for performing accurate intelligent detection of the vertices of the box.

[0005] Therefore, it is necessary to propose a new technical solution to solve the above technical problems. [Summary of the invention]

[0006] The object of the present invention is to provide a method and device for detecting key points of a box, a computing device, and a computer-readable storage medium, which can perform accurate and intelligent detection of key points of a box.

[0007] To solve the above problems, the technical solutions of the embodiments of the present invention are as follows:

[0008] In a first aspect, a box key point detection method is provided, the method comprising the following steps: acquiring first image data, wherein the first image data includes data of an image of a first box; detecting the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of a first key point of the first box; acquiring second image data of a predetermined size corresponding to the first key point from the first image data; detecting the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point.

[0009] In a second aspect, a box key point detection device is provided, the device comprising: a first acquisition module, used to acquire first image data, wherein the first image data includes data of an image of a first box; a detection module, used to detect the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of a first key point of the first box; a second acquisition module, used to acquire second image data of a predetermined size corresponding to the first key point from the first image data; the detection module is also used to detect the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point.

[0010] In a third aspect, a computing device is provided, comprising a processor and a memory, wherein the memory is used to store program instructions. When the computing device is running, the processor is used to execute the program instructions in the memory to execute the box key point detection method described in the first aspect.

[0011] In a fourth aspect, a computer-readable storage medium storing program instructions is provided, wherein the program instructions are used to enable a computer to execute the box key point detection method described in the first aspect.

[0012] Compared with the prior art, the embodiment of the present invention adopts a cascade network model to perform fine screening on the first key points of the first box, that is, the first image data of the first box is first detected using the first detection network model based on posture detection technology to output the first coordinate data of the first key points of the first box, and then the second image data including the first key points is detected using the second detection network model based on face detection technology to output the second coordinate data of the first key points of the first box, thereby obtaining a more accurate position of the key points of the box. It has been verified in practice that the technical solution of the embodiment of the present invention can control the position offset (coordinate error) of the first key point detected to within 3 pixels.

[0013] In order to make the above contents of the present invention more clearly understood, the following is a detailed description of the preferred embodiments in conjunction with the accompanying drawings as follows:

Brief Description of the Drawings

[0014] Figure 1 Schematic diagram of the operating environment of the cabinet key point detection device and method according to an embodiment of the present invention.

[0015] Figure 2 Flow chart of a box key point detection method according to an embodiment of the present invention.

[0016] Figure 3 The present invention is a flowchart of the steps of training the first detection network model in the box key point detection method according to an embodiment of the present invention.

[0017] Figure 4 The present invention is a flowchart of the steps of training the second detection network model in the box key point detection method according to an embodiment of the present invention.

[0018] Figure 5 4 is a block diagram of a box key point detection device according to an embodiment of the present invention. [Specific implementation method]

[0019] In the embodiments of the present invention, the term "module" generally refers to: hardware, a combination of hardware and software, software, etc. For example, a module can be a process running on a processor, a processor, an object, an executable application, an execution thread, a program, etc. Both the application running on the processor and the processor can be modules. One or more modules can be located in one computer and / or distributed between two or more computers.

[0020] refer to Figure 1 , Figure 1 Schematic diagram of the operating environment of the cabinet key point detection device and method according to an embodiment of the present invention.

[0021] Application examples of the box key point detection device of the embodiment of the present invention include but are not limited to personal computers, servers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multi-processor systems, consumer electronic devices, etc.

[0022] The box key point detection device and method of the embodiment of the present invention can be run in the equipment and system corresponding to the above-mentioned application example. The equipment and system corresponding to the application example may include any combination of processor 103, memory 101, interface circuit 102, camera 104, communication circuit 105, etc. Any combination of the above-mentioned processor 103, memory 101, interface circuit 102, camera 104, communication circuit 105, etc. is used to implement the functions and steps of the box key point detection device and method of the embodiment of the present invention.

[0023] refer to Figure 2 , Figure 2 Flow chart of a box key point detection method according to an embodiment of the present invention.

[0024] The box key point detection method of the embodiment of the present invention comprises the following steps:

[0025] Step 201, obtaining first image data, wherein the first image data includes data of an image of a first box (first image). Specifically, step 201 includes: taking an image of the first box to obtain the first image data. The first box is a target box (box to be detected).

[0026] Step 202: Detect the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of a first key point (e.g., vertex) of the first box and a first serial number of the first key point relative to the first box. In the process of detecting the first key point, the detected first key point is numbered (marked with a first serial number). The combination of the first serial number and the first coordinate data is used to determine the length, width, and height data of the first box (the length of three different edges with the same first key point). For the method of determining the length, width, and height data of the first box by combining the first serial number with the first coordinate data, an example is as follows: take any first key point (serial number A) as the starting point, and find the other three first key points (serial number B, serial number C, serial number D) in a clockwise or counterclockwise direction. At this time, the connecting line between the first first key point (serial number B) and the first key point (serial number A) among the three first key points constitutes the first edge, and the third first key point (serial number D) among the three first key points constitutes the first edge. The connecting line between the first key point (serial number A) and the first key point (serial number A) constitutes a second edge. Taking the first key point (serial number A) as the starting point, find the other three first key points (serial number E, serial number F, serial number G) in a clockwise or counterclockwise direction. At this time, the connecting line between the first key point (serial number G) and the first key point (serial number A) constitutes a third edge. Through the coordinate data of these three edges (connecting lines) and the four first key points (serial number A, serial number B, serial number D, serial number G), the length, width and height data of the first box can be calculated. Specifically, the step 202 includes: calling the first detection network model, controlling the first detection network model to receive the first image data, and controlling the first detection network model to detect the first image data to output the first detection result data.

[0027] The first detection network model is a model created based on posture detection technology. The input (received) of the first detection network model is the first image data, and the output of the first detection network model is the first coordinate data of the first key point of the first box. Of course, the output of the first detection network model can also include the connection line data between the first key points of the first box, and the first detection network model is based on posture detection technology as the core, wherein the posture detection-based technology can be, for example, openpose-based technology, DeepCut-based technology, and RMPE (Regional Multi-person Pose Estimation)-based technology. Among them, it is preferred to adopt the technology based on OpenPose. The first detection network model created based on the technology based on OpenPose can achieve higher efficiency and stability in detecting the first image data compared with the model created based on other technologies. The reasons are: First, the first detection network model created based on the technology based on OpenPose can directly detect the key points (vertices) of the box without detecting (deriving) the key points through detecting other features of the box (indirect approach); Second, the first detection network model created based on the technology based on OpenPose does not need to outline the contour (image) of the box before detecting the key points. On the one hand, it can save the step of outlining the contour (image) of the box and improve the efficiency of detecting key points. On the other hand, it can avoid the failure to detect the key points due to the failure of outlining the contour (image) of the box, which is beneficial to improve the stability of detecting the key points of the box.

[0028] That is, the first detection network model detects the key points in the first image data (equivalent to the joints at the ends of the human arm) and the connecting lines between the key points (equivalent to the torso of the human arm) through posture detection-based technology. The connecting line data is represented in the form of a vector, and outputs the first coordinate data of the first key point of the first box.

[0029] Step 203, obtaining second image data of a predetermined size corresponding to the first key point from the first image data. Specifically, step 203 includes: based on the first coordinate data of the first key point, intercepting a second image of the predetermined size including the first key point from the first image corresponding to the first image data, and outputting the second image data corresponding to the second image. For example, intercepting a rectangular second image with the coordinates corresponding to the first coordinate data as the center.

[0030] The predetermined size is a size of X pixels*Y pixels, wherein X and Y may be the same or different. For example, the predetermined size is 20 pixels*20 pixels.

[0031] Step 204: Detect the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point.

[0032] The second detection network model is a model created based on face detection technology.

[0033] The input (received) of the second detection network model is the second image data, the output of the second detection network model is the second coordinate data of the first key point of the first box, and the second detection network model is centered on the technology based on face detection, wherein the technology based on face detection can be, for example, a technology based on multi-task convolutional neural networks (MTCNN, Multi-Task Convolutional Neural Networks) or a technology based on Facenet.

[0034] Specifically, the step 204 includes: calling the second detection network model, controlling the second detection network model to receive the first coordinate data of the first key point, and controlling the second detection network model to detect the second image data based on the first coordinate data to output the second detection result data.

[0035] Among them, the second detection network model performs feature point (key point) regression based on the first coordinate data to output the second detection result data that is more accurate than the first detection result data.

[0036] Furthermore, the second detection network model includes an iterative algorithm. The second detection network model performs an iterative operation based on a previous detection result to generate a more accurate detection result.

[0037] The first coordinate data and the second coordinate data are both two-dimensional coordinate data or three-dimensional coordinate data. The accuracy of the second coordinate data is higher than the accuracy of the first coordinate data.

[0038] Scenario 1: Take an image of the first box to obtain the first image data, receive the first image data, call the first detection network model, input the first image data into the first detection network model, and control the first detection network model (based on posture detection technology) to detect the first image data so that the first detection network model outputs two-dimensional coordinate data of the vertices of the first box (with a first accuracy), crop out second image data of a 50 pixel * 50 pixel second image including the first key point from the first image data, call the second detection network model, and control the second detection network model (based on face detection technology) to detect the second image data so that the second detection network model outputs two-dimensional coordinate data of the vertices of the first box (with a second accuracy), wherein the second accuracy is higher than the first accuracy.

[0039] Scenario 2: Receive the first image data taken, input the first image data into the first detection network model, and control the first detection network model (based on posture detection technology) to detect the first image data so that the first detection network model outputs the first coordinate data of the first key point (with a first accuracy), captures a second image with a size of 40 pixels * 30 pixels including the first key point from the first image corresponding to the first image data, and outputs the second image data corresponding to the second image, inputs the second image data into the second detection network model, and controls the second detection network model (based on face detection technology) to detect the first key point based on the second image data, so that the second detection network model outputs the second coordinate data of the first key point (with a second accuracy), wherein the second accuracy is higher than the first accuracy.

[0040] Scenario three: Receive the first image data, control the first detection network model to receive the first image data, and control the first detection network model (based on posture detection technology) to detect the first image data, so that the first detection network model outputs the first coordinate data of the first key point and the connecting line data of the first key point, and obtains a 20 pixel * 20 pixel second image including the first key point from the first image corresponding to the first image data with the point corresponding to the first coordinate data as the center, and outputs the second image data corresponding to the second image, and controls the second detection network model (based on face detection technology) to detect the second image data according to at least one of the first coordinate data of the first key point and the connecting line data of the first key point, so that the second detection network model outputs the second coordinate data of the first key point.

[0041] refer to Figure 3 , Figure 3 The present invention is a flowchart of the steps of training the first detection network model in the box key point detection method according to an embodiment of the present invention.

[0042] Before the step of acquiring the first image data, the box key point detection method further includes the following steps:

[0043] Step 301: Import the first training data prepared in advance.

[0044] Step 302: Train the first detection network model using first training data, wherein the first training data includes third image data of the second box and annotated data for second key points of the second box.

[0045] The second box may be a box different from the first box, and the image data of the second box and the labeled data of the second key points of the second box are used for training purposes.

[0046] refer to Figure 4 , Figure 4 The present invention is a flowchart of the steps of training the second detection network model in the box key point detection method according to an embodiment of the present invention.

[0047] Before the step of detecting the second image data using the second detection network model to output second detection result data, the box key point detection method further includes the following steps:

[0048] Step 401: Import the second training data prepared in advance.

[0049] Step 402: train the second detection network model using second training data, wherein the second training data includes image data obtained based on third coordinate data of a third key point of a third box obtained in advance through training.

[0050] The third coordinate data may be data outputted after training with the first detection network model, or may be coordinate data of key points produced separately.

[0051] The third box and the second box may be the same box or different boxes.

[0052] In the case where the data provided to the second detection network model for training is image data obtained based on the coordinate data of the second key point output by the first detection network model (i.e., the third box and the second box are the same box), the box key point detection method further includes the following steps:

[0053] During the training of the first detection network model, the first detection network model is controlled to mark the second key point with a second serial number. For example, during the training process, the first detection network model first detects the second key point located at the upper left corner of the first image, and marks the second key point located at the upper left corner of the first image as serial number 1, and then the first detection network model detects the second key point located at the lower left corner of the first image, and marks the second key point located at the lower left corner of the first image as serial number 2, and so on.

[0054] In the case where the data provided to the second detection network model for training is not image data obtained based on the coordinate data of the second key point output by the first detection network model (i.e., the process of the first training module training the first detection network model is independent of the process of the second training module training the second detection network model, and the third box and the second box are different boxes), the box key point detection method further includes the following steps:

[0055] The second detection network model is trained independently using second training data.

[0056] In the embodiment of the present invention, the step of training the first detection network model and the step of training the second detection network model can be performed independently or in a sequential order.

[0057] Scenario 4: independently executing the step of training the first detection network model, independently executing the step of training the second detection network model, and then independently executing step 201, step 202, step 203 and step 204.

[0058] Scenario 5: First execute the step of training the first detection network model, then execute the step of training the second detection network model, and then execute step 201, step 202, step 203 and step 204.

[0059] In the above technical scheme, since the first key points of the first box are finely screened by using a cascade network model, that is, the first image data of the first box is first detected by using the first detection network model based on posture detection technology to output the first coordinate data of the first key points of the first box, and then the second image data including the first key points is detected by using the second detection network model based on face detection technology to output the second coordinate data of the first key points of the first box, so that a more accurate position of the key points of the box can be obtained. It has been verified in practice that the technical scheme of the embodiment of the present invention can control the position offset (coordinate error) of the first key point detected to within 3 pixels.

[0060] refer to Figure 5 , Figure 5 4 is a block diagram of a box key point detection device according to an embodiment of the present invention.

[0061] The box key point detection device implemented in the present invention includes a first acquisition module 501 , a detection module 502 and a second acquisition module 503 .

[0062] The first acquisition module 501 is used to acquire first image data, wherein the first image data includes data of an image of a first box. The first acquisition module 501 is used to capture an image of the first box to acquire the first image data. In this case, the first acquisition module 501 may be, for example, a camera integrated in the box key point detection device; or, the first acquisition module 501 is used to receive the first image data from a camera integrated in the box key point detection device. In this case, the first acquisition module 501 may be, for example, an interface circuit in a main board; or, the first acquisition module 501 is used to receive the first image data from other camera devices that communicate with the box key point detection device. In this case, the first acquisition module 501 may be, for example, a communication circuit / signal transceiver circuit. The first box is a target box (box to be detected).

[0063] The detection module 502 is used to detect the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of a first key point (e.g., vertex) of the first box and a first serial number of the first key point relative to the first box. The detection module 502 is used to number the first key point detected (mark the first serial number) during the detection of the first key point. The combination of the first serial number and the first coordinate data is used to determine the length, width, and height data of the first box (the length of three different edges with the same first key point). For the method of determining the length, width, and height data of the first box by combining the first serial number with the first coordinate data, an example is as follows: taking any first key point (serial number A) as the starting point, find the other three first key points (serial number B, serial number C, serial number D) in a clockwise or counterclockwise direction. At this time, the connecting line between the first first key point (serial number B) and the first key point (serial number A) among the three first key points constitutes the first edge, and the third first key point (serial number D) among the three first key points constitutes the first edge. The connecting line between the first key point (serial number A) and the first key point constitutes the second edge. Taking the first key point (serial number A) as the starting point, find the other three first key points (serial number E, serial number F, serial number G) in a clockwise or counterclockwise direction. At this time, the connecting line between the first key point (serial number G) and the first key point (serial number A) constitutes the third edge. Through the coordinate data of these three edges (connecting lines) and the four first key points (serial number A, serial number B, serial number D, serial number G), the length, width and height data of the first box can be calculated.

[0064] The detection module 502 is used to call the first detection network model, control the first detection network model to receive the first image data, and control the first detection network model to detect the first image data to output the first detection result data.

[0065] The first detection network model is a model created based on posture detection technology. The input (received) of the first detection network model is the first image data, and the output of the first detection network model is the first coordinate data of the first key point of the first box. Of course, the output of the first detection network model can also include the connection line data between the first key points of the first box, and the first detection network model is based on posture detection technology as the core, wherein the posture detection-based technology can be, for example, openpose-based technology, DeepCut-based technology, and RMPE (Regional Multi-person Pose Estimation)-based technology. Among them, the openpose-based technology is preferably used.

[0066] That is, the first detection network model is used to detect key points in the first image data (equivalent to the joints at the ends of the human arm) and connecting lines between the key points (equivalent to the torso of the human arm) through posture detection-based technology. The connecting line data is represented in the form of a vector, and the first coordinate data of the first key point of the first box is output.

[0067] The second acquisition module 503 is used to acquire second image data of a predetermined size corresponding to the first key point from the first image data.

[0068] The second acquisition module 503 is used to intercept a second image having the predetermined size and including the first key point from the first image corresponding to the first image data based on the first coordinate data of the first key point, and output the second image data corresponding to the second image. For example, the second acquisition module 503 is used to intercept a rectangular second image centered on the coordinates corresponding to the first coordinate data.

[0069] The predetermined size is a size of X pixels*Y pixels, wherein X and Y may be the same or different. For example, the predetermined size is 20 pixels*20 pixels.

[0070] The second acquisition module 503 may be, for example, a program or an image processing program for implementing a screenshot function in computer software (eg, input method software, browser software, instant messaging software), or an image processing chip.

[0071] The detection module 502 is further used to detect the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point.

[0072] The second detection network model is a model created based on face detection technology.

[0073] The input (received) of the second detection network model is the second image data, the output of the second detection network model is the second coordinate data of the first key point of the first box, and the second detection network model is centered on the technology based on face detection, wherein the technology based on face detection can be, for example, a technology based on multi-task convolutional neural networks (MTCNN, Multi-Task Convolutional Neural Networks) or a technology based on Facenet.

[0074] The detection module 502 is used to call the second detection network model, and control the second detection network model to receive the first coordinate data of the first key point, and control the second detection network model to detect the second image data based on the first coordinate data to output the second detection result data.

[0075] The detection module 502 is used to control the second detection network model to perform feature point (key point) regression based on the first coordinate data to output the second detection result data that is more accurate than the first detection result data.

[0076] Furthermore, the second detection network model includes an iterative algorithm. The second detection network model is used to perform an iterative operation according to a previous detection result to generate a more accurate detection result.

[0077] The first coordinate data and the second coordinate data are both two-dimensional coordinate data or three-dimensional coordinate data. The accuracy of the second coordinate data is higher than the accuracy of the first coordinate data.

[0078] Scenario 1: The camera (the first acquisition module 501) integrated in the computer (the box key point detection device) captures the image of the first box to obtain the first image data. The operating system program (the detection module 502) run by the processor of the computer receives the first image data, calls the first detection network model, inputs the first image data into the first detection network model, and controls the first detection network model (based on the posture detection technology) to detect the first image data, so that the first detection network model outputs the two-dimensional coordinate data of the vertices of the first box (with the first accuracy). The image processing program (the second acquisition module 503) run by the processor of the computer cuts out the second image data of the second image of 50 pixels * 50 pixels including the first key point from the first image data. The operating system program (the detection module 502) run by the processor of the computer calls the second detection network model, and controls the second detection network model (based on the face detection technology) to detect the second image data, so that the second detection network model outputs the two-dimensional coordinate data of the vertices of the first box (with the second accuracy), wherein the second accuracy is higher than the first accuracy.

[0079] Scenario 2: The interface circuit (the first acquisition module 501) in the mainboard of the mobile terminal (the box key point detection device) receives the first image data taken by the camera from the camera integrated in the mobile terminal. The operating system program (the detection module 502) of the mobile terminal inputs the first image data into the first detection network model, and controls the first detection network model (based on the posture detection technology) to detect the first image data, so that the first detection network model outputs the first coordinate data (with the first accuracy) of the first key point. The processor of the mobile terminal runs the screenshot program (the second acquisition module 503), captures the second image with a size of 40 pixels * 30 pixels including the first key point from the first image corresponding to the first image data, and outputs the second image data corresponding to the second image. The operating system program (the detection module 502) of the mobile terminal inputs the second image data into the second detection network model, and controls the second detection network model (based on the face detection technology) to detect the first key point based on the second image data, so that the second detection network model outputs the second coordinate data (with the second accuracy) of the first key point, wherein the second accuracy is higher than the first accuracy.

[0080] Scenario 3: The communication circuit (the first acquisition module 501) of the server (the box key point detection device) receives the first image data from other camera devices communicating with the server. The processor of the server runs the detection control program (the detection module 502), controls the first detection network model to receive the first image data, and controls the first detection network model (based on the posture detection technology) to detect the first image data, so that the first detection network model outputs the first coordinate data of the first key point and the connecting line data of the first key point. The image processor (GPU, Graphics Processing Unit) of the server runs the image processing program (the second acquisition module 503) to obtain a second image of 20 pixels * 20 pixels including the first key point from the first image corresponding to the first image data with the point corresponding to the first coordinate data as the center, and outputs the second image data corresponding to the second image. The processor of the server runs the detection control program (the detection module 502), controls the second detection network model (based on the face detection technology) to detect the second image data according to at least one of the first coordinate data of the first key point and the connecting line data of the first key point, so that the second detection network model outputs the second coordinate data of the first key point.

[0081] The box key point detection device also includes a first training module, which is used to import the first training data prepared in advance and to train the first detection network model using the first training data, wherein the first training data includes third image data of the second box and annotated data for the second key point of the second box. The second box may be a box different from the first box, and the image data of the second box and the annotated data for the second key point of the second box are used for training purposes.

[0082] The first training module may be, for example, a deep learning platform based on CAFFE (Convolutional Architecture for Fast Feature Embedding, convolutional neural network framework).

[0083] The box key point detection device also includes a second training module, which is used to import the second training data prepared in advance and to train the second detection network model using the second training data, wherein the second training data includes image data obtained based on the third coordinate data of the third key point of the third box obtained in advance through training. The third coordinate data can be the data output by the first detection network model after training, or it can be the coordinate data of the key point produced separately.

[0084] The third box and the second box may be the same box or different boxes.

[0085] Similarly, the second training module may be, for example, a deep learning platform based on CAFFE. The second training module and the first training module may be the same module, or may be separate and different modules.

[0086] When the data provided to the second detection network model for training is image data acquired based on the coordinate data of the second key point output by the first detection network model (i.e., the third box and the second box are the same box), the first training module controls the first detection network model to mark the second key point with a second serial number during the process of training the first detection network model.

[0087] For example, during the training process, the first detection network model first detects the second key point located at the upper left corner of the first image, and marks the second key point located at the upper left corner of the first image as serial number 1, and the first detection network model then detects the second key point located at the lower left corner of the first image, and marks the second key point located at the lower left corner of the first image as serial number 2, and so on.

[0088] When the data provided to the second detection network model for training is not image data obtained based on the coordinate data of the second key point output by the first detection network model (that is, the process of training the first detection network model by the first training module is independent of the process of training the second detection network model by the second training module, and the third box and the second box are different boxes), the second training module independently trains the second detection network model using the second training data.

[0089] The first training module and the second training module may be expressed as one or a combination of more than one of an operating system program, a training control program, a deep learning platform program, etc., run by a processor.

[0090] In the above technical scheme, since the first key points of the first box are finely screened by using a cascade network model, that is, the first image data of the first box is first detected by using the first detection network model based on posture detection technology to output the first coordinate data of the first key points of the first box, and then the second image data including the first key points is detected by using the second detection network model based on face detection technology to output the second coordinate data of the first key points of the first box, so that a more accurate position of the key points of the box can be obtained. It has been verified in practice that the technical scheme of the embodiment of the present invention can control the position offset (coordinate error) of the first key point detected to within 3 pixels.

[0091] The box key point detection device implemented by the present invention can be implemented by a CPU (Central Processing Unit), a GPU, an NPU (Neural Processing Unit), other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0092] The box key point detection device implemented in the present invention can also be implemented by software. In this case, the box key point detection device implemented in the present invention and its various modules can also be software modules.

[0093] The box key point detection device implemented in the present invention may correspond to the method described in the embodiment of the present invention, and the above and other operations and / or functions of each module in the box key point detection device are used to implement the corresponding process of the box key point detection method implemented in the present invention.

[0094] The computing device of the embodiment of the present invention includes a processor and a memory. The processor and the memory communicate via a bus. The memory is used to store program instructions. When the computing device is running, the processor is used to call the program instructions stored in the memory, and is used to execute the program instructions to execute the box key point detection method implemented by the present invention. The computing device may be, for example: a personal computer, a server, a handheld or laptop device, a mobile device (including: a mobile phone, a personal digital assistant, a media player, etc.), a multi-processor system, a consumer electronic device, etc. For example, the computing device is a handheld scanner for couriers / logistics operators, or it may be a server set in a logistics warehouse for automatic identification of circulating boxes.

[0095] It should be understood that the processor may be a CPU, GPU, NPU, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0096] The memory may include a read-only memory and a random access memory, and provide program instructions and data to the processor. The memory may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache.

[0097] The computer-readable storage medium of the embodiment of the present invention stores program instructions, and the program instructions are used to enable a computer to execute the box key point detection method implemented by the present invention.

[0098] The embodiments of the present invention may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product (whose carrier may be, for example, a computer-readable storage medium of an embodiment of the present invention). The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in accordance with the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL, Digital Subscriber Line)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state drive (SSD).

[0099] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application.

[0100] In summary, although the present invention has been disclosed as above in terms of preferred embodiments, the above preferred embodiments are not intended to limit the present invention. A person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined in the claims.

Claims

1. A box key point detection method, characterized in that: The method comprises the following steps: Acquire first image data, wherein the first image data includes data of an image of a first box; Detecting the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of first key points of the first box and connection line data between the first key points, and the first detection network model is a model created based on openpose technology; Acquire second image data of a predetermined size corresponding to the first key point from the first image data; Detecting the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point; The detecting the first image data using the first detection network model to output first detection result data includes: Detecting the first image data using a first detection network model to obtain a first coordinate number of a first key point of the first box, and determining other first key points in a clockwise or counterclockwise direction with the first key point as a starting point; The connecting lines formed by the first key point and other first key points are obtained, so as to calculate the volume of the goods loaded in the first box based on the connecting lines.

2. The box key point detection method according to claim 1, characterized in that: The second detection network model is a model created based on face detection technology.

3. The box key point detection method according to claim 1, characterized in that: The step of acquiring second image data of a predetermined size corresponding to the first key point from the first image data comprises: Based on the first coordinate data of the first key point, a second image with the predetermined size including the first key point is intercepted from the first image corresponding to the first image data, and the second image data corresponding to the second image is output.

4. The box key point detection method according to claim 1, characterized in that: Before the step of acquiring the first image data, the method further comprises the following steps: The first detection network model is trained using first training data, wherein the first training data includes third image data of the second box and annotated data for second key points of the second box.

5. The box key point detection method according to claim 4, characterized in that: The method further comprises the following steps: During the training of the first detection network model, a second serial number is marked for the second key point.

6. The box key point detection method according to claim 1, characterized in that: Before the step of detecting the second image data using the second detection network model to output second detection result data, the method further includes the following steps: The second detection network model is trained using second training data, wherein the second training data includes image data acquired based on third coordinate data of a third key point of a third box obtained in advance through training.

7. A box key point detection device, characterized in that: The device comprises: A first acquisition module, used to acquire first image data, wherein the first image data includes image data of a first box; A detection module, configured to detect the first image data using a first detection network model to output first detection result data, wherein the first detection result data includes first coordinate data of first key points of the first box and connection line data between the first key points, and the first detection network model is a model created based on openpose technology; A second acquisition module, used for acquiring second image data of a predetermined size corresponding to the first key point from the first image data; The detection module is further used to detect the second image data using a second detection network model to output second detection result data, wherein the second detection result data at least includes second coordinate data of the first key point; The detection module is further used for: Detecting the first image data using a first detection network model to obtain a first coordinate number of a first key point of the first box, and determining other first key points in a clockwise or counterclockwise direction with the first key point as a starting point; The connecting lines formed by the first key point and other first key points are obtained, so as to calculate the volume of the goods loaded in the first box based on the connecting lines.

8. The box key point detection device according to claim 7, characterized in that: The second acquisition module is used to capture a second image with the predetermined size including the first key point from the first image corresponding to the first image data based on the first coordinate data of the first key point, and output the second image data corresponding to the second image.

9. A computing device, characterized in that The computing device includes a processor and a memory, the memory is used to store program instructions, and when the computing device is running, the processor is used to execute the program instructions in the memory to execute the box key point detection method according to any one of claims 1 to 6.

10. A computer-readable storage medium storing program instructions, characterized in that: The program instructions are used to enable a computer to execute the box key point detection method described in any one of claims 1 to 6.