Body length detection method and device for target object

Abstract

The application discloses a body length detection method and device of a target object. The method comprises the following steps: acquiring a first image to be detected; performing preprocessing on the first image to obtain a second image, wherein the preprocessing at least comprises image expansion processing; performing analysis and processing on the second image by using a pre-trained target detection model to obtain first detection box information and first depth information of a plurality of first objects in the second image, wherein the first detection box information at least comprises detection box coordinate information and a rotation angle information of the detection box relative to a horizontal line; and determining the body length of each first object according to the first detection box information and the first depth information. The application solves the technical problem that the body length of a target object cannot be simply and efficiently calculated according to an image due to the limitation of camera angle and other conditions in the related art.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a method and apparatus for detecting the body length of a target object. Background Technology

[0002] Currently, automated monitoring systems based on video image and target detection algorithms are widely used in various fields of life. In the field of smart farming, deep learning technology is often used to count and measure pigs. Common target detection algorithms can detect and locate targets in images and classify them to complete the counting task. However, due to issues such as the variable angle of cameras in the pigsty and the large changes in pig body movements and angles, they cannot complete body length measurement. To measure pig body length using image methods, image segmentation methods are needed to classify pig images at the pixel level. However, image segmentation methods suffer from high annotation costs, high requirements for camera angles, and low accuracy when targets are occluded. In addition, there are keypoint detection-based methods for pig body length detection. These methods detect multiple keypoints on the pig's back and concatenate them to estimate the pig's length. However, they suffer from large keypoint localization errors, significant influence from lighting and angle, and severe impact from occlusion. In the implementation of these methods, the distance between the target and the camera affects the judgment results, ultimately leading to the use of markers as references or directly fixing the camera height, resulting in poor universality.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method and apparatus for detecting the body length of a target object, thereby at least solving the technical problem in related technologies where the body length of a target object cannot be calculated simply and efficiently from an image due to limitations such as camera angle.

[0005] According to one aspect of the embodiments of this application, a method for detecting the body length of a target object is provided, comprising: acquiring a first image to be detected; preprocessing the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; analyzing and processing the second image using a pre-trained target detection model to obtain first detection box information and first depth information of a plurality of first objects in the second image, wherein the first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to a horizontal line; and determining the body length of each first object based on the first detection box information and the first depth information.

[0006] Optionally, the first image is preprocessed to obtain the second image, including: determining a pre-defined area to be filled around the first image based on the length and width of the first image; and filling the pre-defined pixel value area with pixels to obtain the second image.

[0007] Optionally, the object detection model includes at least: a feature extraction module, a bounding box prediction module, and a depth prediction module. The pre-trained object detection model is used to analyze and process the second image to obtain first detection box information and first depth information for multiple first objects in the second image. This includes: using the feature extraction module to extract features from the second image to obtain a first feature vector corresponding to the second image; using the bounding box prediction module to analyze and process the first feature vector to obtain first detection box information for multiple first objects in the second image, wherein the detection box coordinate information includes: the coordinates of the center point of the detection box and the width and height information of the detection box; using the depth prediction module to analyze and process the first feature vector to obtain a depth image corresponding to the second image, and determining the first depth information of multiple first objects in the second image based on the depth image, wherein the first depth information includes: a first distance between the center point of the detection box and the target image acquisition device, where the target image acquisition device is the image acquisition device that acquires the first image.

[0008] Optionally, the training process of the object detection model includes: constructing an initial detection model; acquiring multiple third images from image acquisition devices at various acquisition points under various lighting conditions, preprocessing each third image to obtain multiple fourth images; determining the annotation information corresponding to each fourth image, wherein the annotation information includes: the second detection box information and the second depth information of each second object in the fourth image; and iteratively training the initial detection model using multiple fourth images and annotation information to obtain the object detection model.

[0009] Optionally, determining the annotation information corresponding to each fourth image includes: for each fourth image, acquiring the second depth information corresponding to the fourth image acquired by a depth data acquisition device at the same acquisition point and frequency as the image acquisition device that acquired the fourth image; for each second object in the fourth image, determining a rectangular detection box that is parallel to and fits the body of the second object, and recording the coordinate information of the rectangular detection box and the rotation angle information between the long side of the rectangular detection box and the horizontal line as the second detection box information of the second object.

[0010] Optionally, the body length of each first object is determined based on the detection box information and depth information, including: determining the second length of a reference object in the first image whose actual length is the first length, and determining the ratio of the first length to the second length as a mapping parameter; for each first object in the second image, the body length of the first object is determined based on the first detection box information, the first depth information and the mapping parameter corresponding to the first object.

[0011] Optionally, determining the body length of the first object based on the first detection box information, first depth information, and mapping parameters corresponding to the first object includes: determining a second distance between the center point of the reference object and the target image acquisition device based on the depth image, and determining a third length of the longer side of the detection box corresponding to the first object; determining the product of the first distance, the third length, and the mapping parameters, and determining the ratio of the product to the second distance as the body length of the first object.

[0012] According to another aspect of the embodiments of this application, a target object body length detection device is also provided, comprising: an acquisition module for acquiring a first image to be detected; a preprocessing module for preprocessing the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; a detection module for analyzing and processing the second image using a pre-trained target detection model to obtain first detection box information and first depth information of a plurality of first objects in the second image, wherein the first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line; and a determination module for determining the body length of each first object based on the first detection box information and the first depth information.

[0013] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described target object length detection method by running the computer program.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described target object length detection method through the computer program.

[0015] In this embodiment, after acquiring the first image to be detected, preprocessing operations, including image expansion, are performed on the first image to obtain a second image. Then, a pre-trained target detection model is used to analyze and process the second image to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line. Finally, the body length of each first object is determined based on the first detection box information and the first depth information. By introducing rotation angle information, the rectangular detection box is rotated to better fit the body of the target object, which solves the problem in traditional target detection where parallel axis rectangular boxes cannot measure body length due to changes in camera angle and target object body shape. At the same time, by introducing depth prediction to determine the distance from the target object to the camera, and then calculating the accurate length of the target object together with the detection box information, the problem of inconsistent prediction of different target scales due to distance can be solved. The solution of this application can simply and efficiently complete the body length measurement of the target object, and can support any camera angle, effectively solving the technical problem in related technologies where the body length of the target object cannot be simply and efficiently calculated from the image due to limitations such as camera angle. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a schematic diagram of the structure of an optional computer terminal according to an embodiment of this application;

[0018] Figure 2 This is a flowchart illustrating an optional target object length detection method according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of the network structure of an optional depth prediction module according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of an optional image preprocessing and annotation method according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of an optional method for determining camera mapping parameters according to an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of an optional pig body length detection process according to an embodiment of this application;

[0023] Figure 7This is a schematic diagram of the structure of an optional target object length detection device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] According to an embodiment of this application, a method for detecting the body length of a target object is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for detecting the body length of a target object is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the target object length detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned application vulnerability detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0032] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0033] Under the above operating environment, this application provides a method for detecting the body length of a target object, such as... Figure 2 As shown, the method includes the following steps:

[0034] Step S202: Obtain the first image to be detected;

[0035] Step S204: Preprocess the first image to obtain the second image, wherein the preprocessing includes at least: image expansion processing;

[0036] Step S206: Analyze and process the second image using a pre-trained object detection model to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and detection box rotation angle information relative to the horizontal line.

[0037] Step S208: Determine the body length of each first object based on the first detection frame information and the first depth information.

[0038] It should be noted that the target object body length detection method in this application embodiment is mainly applied to the field of smart farming. The target object can be livestock such as pigs, cattle, and sheep. The following uses pigs as an example and describes each step of the target object body length detection method in conjunction with the specific implementation process.

[0039] As an optional implementation, in order to accurately analyze and obtain the detection bounding box information and depth information of the target object in the image, the target detection model can be trained in the following way:

[0040] First, an initial detection model is constructed, which includes at least the following modules: feature extraction, bounding box prediction, and depth prediction.

[0041] Optionally, the feature extraction module can use a CNN (Convolutional Neural Network) structure such as ResNet50 or EfficientNet to extract feature information from the input image. Its output is a vector with dimensions (N, W0, H0, C0), where N is the batch size of the training batch, W0 and H0 are the width and height of the feature vector, and C0 is the number of output channels. Each parameter can be adjusted according to actual needs, and no specific limitation is made here.

[0042] The bounding box prediction module can adopt the prediction module structure from object detection algorithms such as SSD (Single Shot Multibox Detector) and YOLO (You Only Look Once) series, and add an angle prediction branch to the original structure, adding a new dimension for angle prediction to the output detection result channels. Taking the prediction module in YOLOv5 as an example, under normal circumstances, the number of output channels of YOLOv5 at one scale is "number of detection boxes * (number of categories + foreground probability + bounding box information)". Assuming the number of categories is 80 and the number of detection boxes is 3, the number of output channels is 3 * (80 + 1 + 4) = 255. After modification, an additional angle prediction is performed for each detection box, and the number of output channels is now "number of detection boxes * (number of categories + foreground probability + bounding box information + angle)". If the number of categories is 80 and the number of detection boxes is 3, the number of output channels is 3 * (80 + 1 + 4 + 1) = 258. During training, the rotation angles from 0 to 180° need to be normalized and mapped to 0 to 1 for training. Finally, the predicted results are restored to the true angles through inverse normalization.

[0043] The depth prediction module can consist of multiple convolutional layers and a final fully connected layer, such as... Figure 3 As shown, the number of channels in the network increases from shallow to deep and then from deep to shallow, and the width and height decrease from large to small and then increase from small to large. The network depth can be adjusted automatically. Finally, the output vector of the fully connected layer has dimensions (N, W1, H1), where N is the size of the training batch size, and W1 and H1 are the width and height of the output depth image, which are the same as the width and height of the input image. Each pixel value of the depth image represents the predicted distance from that point to the camera.

[0044] Then, training samples for model training can be obtained as follows: acquire multiple third images acquired by image acquisition devices at multiple acquisition points under various lighting conditions, preprocess each third image to obtain multiple fourth images; determine the annotation information corresponding to each fourth image, wherein the annotation information includes: the second detection box information and the second depth information of each second object in the fourth image.

[0045] Specifically, in order to enhance the richness of the sample and improve the model training and prediction effects, considering the different light changes in farms during the day and night, third images can be collected over multiple days from cameras at different heights and positions under different daytime and nighttime conditions. The collected third images should contain as many pigs of different colors, types and sizes as possible, and the pigs in the images should be clearly visible. The total amount of image data should preferably be no less than 10,000 images.

[0046] Since the detection boxes in this application have a rotation angle, some detection boxes may extend beyond the range of the original third image. To avoid this situation, all acquired third images can be preprocessed.

[0047] Optionally, the third image can be expanded as follows: A pre-defined area to be filled around the third image is determined based on its length and width. This area is then filled with pixels of a pre-defined value to obtain the corresponding fourth image. For example, when filling the perimeter of a third image, 10% of the image's height is filled vertically, and 10% of its width is filled horizontally. The three-channel pixel values ​​of the filled pixels are all 127.

[0048] Afterwards, image enhancement methods, such as mosaic enhancement and scaling, can be used to enlarge, reduce, or stitch the fourth image to enhance its diversity.

[0049] After preprocessing, the annotation information corresponding to each fourth image can be determined in the following way: For each fourth image, the second depth information corresponding to the fourth image can be obtained from the depth data acquisition device that has the same acquisition point and frequency as the image acquisition device that acquired the fourth image. The depth data acquisition device can be a TOF (Time of Flight) sensor, etc. For each second object in the fourth image, a rectangular detection box that is parallel to and fits the body of the second object is determined, and the coordinate information of the rectangular detection box and the rotation angle information between the long side of the rectangular detection box and the horizontal line are recorded as the second detection box information of the second object.

[0050] When determining the detection frame, you can first determine the parallel axis rectangle containing the second object, and then rotate the rectangle so that the rotated rectangle fits as closely as possible to the body of the second object. Then record the five parameters: the center point coordinates (x, y) of the rotated rectangle, the width w of the rectangle, the height h of the rectangle, and the clockwise rotation angle α around the center point.

[0051] Figure 4A schematic diagram of an optional image preprocessing and annotation is shown, wherein (a) is the original third image, (b) is the fourth image after image expansion processing of the third image, and (c) is the annotated fourth image.

[0052] Finally, multiple fourth images are used as training samples, and the initial detection model is iteratively trained using multiple training samples and their annotation information to obtain the target detection model.

[0053] In practical applications, the first image to be detected is first acquired, and then the first image can be preprocessed in the following way to obtain the second image: the area to be filled around the first image is determined according to the length and width of the first image; the area to be filled is filled with pixels of a preset pixel value to obtain the second image.

[0054] Then, the pre-trained object detection model can be used to analyze and process the second image to obtain the first detection box information and first depth information of multiple first objects in the second image.

[0055] Specifically, the feature extraction module can first extract features from the second image to obtain a first feature vector corresponding to the second image; then, the bounding box prediction module can analyze and process the first feature vector to obtain first detection box information for multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line. The detection box coordinate information includes: center point coordinate information and width and height information of the detection box; then, the depth prediction module can analyze and process the first feature vector to obtain a depth image corresponding to the second image, and determine the first depth information of multiple first objects in the second image based on the depth image. The first depth information includes: the first distance between the center point of the detection box and the target image acquisition device, where the target image acquisition device is the image acquisition device that acquires the first image.

[0056] As an optional implementation, when determining the body length of each first object based on the detection box information and depth information, the second length of a reference object in the first image whose actual length is the first length can be determined first, and the ratio of the first length to the second length can be determined as a mapping parameter. The reference object can be an object with a standard length, such as a fence in the image. Then, for each first object in the second image, the body length of the first object is determined based on the first detection box information, the first depth information and the mapping parameter corresponding to the first object.

[0057] Optionally, for each first object in the second image, a second distance between the center point of the reference object and the target image acquisition device can be determined based on the depth image, and a third length of the longer side of the detection box corresponding to the first object can be determined; the product of the first distance, the third length and the mapping parameter can be determined, and the ratio of the product to the second distance can be determined as the body length of the first object.

[0058] by Figure 5 For example, if the first length of the actual object used as a reference is L1 (in centimeters), and the second length of the object in the first image captured by the camera is L2 (in pixels), then the mapping parameter corresponding to the camera is β = L1 / L2. If the second distance between the object and the camera is determined to be D (in centimeters) based on the depth image, the first distance between the center point of the detection box corresponding to the first object and the camera is d (in centimeters), and the third length of the longer side of the detection box is l (in pixels), then the body length of the first object is determined to be... (Unit: centimeters)

[0059] Figure 6 This is a schematic diagram of a complete pig body length detection process according to an embodiment of this application. First, the acquired pig image is input into a feature extraction module with a CNN structure to obtain a first feature vector corresponding to the image. Then, the bounding box prediction module is used to perform angle prediction, bounding box prediction, and center point prediction on the first feature vector to obtain the rotation angle α, width and height (w, h), and center point (x, y) of the detection box for each pig in the image. At the same time, the depth prediction module is used to determine the first distance d between the center point of the detection box and the camera. The body length of each pig is calculated based on the output results of the bounding box prediction module and the depth prediction module.

[0060] In this embodiment, after acquiring the first image to be detected, preprocessing operations, including image expansion, are performed on the first image to obtain a second image. Then, a pre-trained target detection model is used to analyze and process the second image to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line. Finally, the body length of each first object is determined based on the first detection box information and the first depth information. By introducing rotation angle information, the rectangular detection box is rotated to better fit the body of the target object, which solves the problem in traditional target detection where parallel axis rectangular boxes cannot measure body length due to changes in camera angle and target object body shape. At the same time, by introducing depth prediction to determine the distance from the target object to the camera, and then calculating the accurate length of the target object together with the detection box information, the problem of inconsistent prediction of different target scales due to distance can be solved. The solution of this application can simply and efficiently complete the body length measurement of the target object, and can support any camera angle, effectively solving the technical problem in related technologies where the body length of the target object cannot be simply and efficiently calculated from the image due to limitations such as camera angle.

[0061] Example 2

[0062] According to an embodiment of this application, a target object length detection device for implementing the target object length detection method in Embodiment 1 is also provided, such as... Figure 7 As shown, the target object length detection device includes at least: an acquisition module 71, a preprocessing module 72, a detection module 73, and a determination module 74, wherein:

[0063] Acquisition module 71 is used to acquire the first image to be detected;

[0064] Preprocessing module 72 is used to preprocess the first image to obtain the second image, wherein the preprocessing includes at least: image expansion processing;

[0065] The detection module 73 is used to analyze and process the second image using a pre-trained target detection model to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and detection box rotation angle information relative to the horizontal line.

[0066] The determination module 74 is used to determine the body length of each first object based on the first detection frame information and the first depth information.

[0067] Optionally, the target object length detection device in this application embodiment further includes a model training module for training a target detection model.

[0068] The following describes the functions of each module of the target object length detection device in conjunction with specific implementation methods.

[0069] As an optional implementation, in order to accurately analyze and obtain the detection bounding box information and depth information of the target object in the image, the model training module can first train the target detection model in the following way:

[0070] First, an initial detection model is constructed, which includes at least three modules: a feature extraction module, a bounding box prediction module, and a depth prediction module. The feature extraction module can use a CNN network such as ResNet50 or EfficientNet to extract feature information from the input image. The bounding box prediction module can use the prediction module structure from object detection algorithms such as SSD or YOLO series, adding an angle prediction branch to the original structure to add a new dimension for angle prediction to the output detection result channel. The depth prediction module can consist of multiple convolutional layers and a final fully connected layer, used to predict the distance from each point in the image to the camera.

[0071] Subsequently, the model training module can obtain training samples for model training in the following way: acquire multiple third images acquired by image acquisition devices at multiple acquisition points under various lighting conditions, preprocess each third image to obtain multiple fourth images; determine the annotation information corresponding to each fourth image, wherein the annotation information includes: the second detection box information and the second depth information of each second object in the fourth image.

[0072] Specifically, in order to enhance the richness of the sample and improve the model training and prediction effects, the model training module can collect third images taken by cameras at different heights and positions under different daytime and nighttime conditions over multiple days, taking into account the different light changes in the farm during the day and night. The collected third images should contain as many pig data of different colors, types and sizes as possible, and the pigs in the images should be clearly visible. The total image data volume should preferably be no less than 10,000 images.

[0073] Since the detection boxes in this application have a rotation angle, some detection boxes may extend beyond the range of the original third image. To avoid this situation, the model training module can preprocess all acquired third images.

[0074] Optionally, the model training module can first perform image expansion processing on the third image in the following way: determine a pre-defined area to be filled around the third image based on its length and width, and fill the area with pixels of a pre-defined pixel value to obtain the corresponding fourth image. For example, when filling the area around a third image, fill 10% of the height of the third image vertically and 10% of the width of the third image horizontally, with the three-channel pixel value of the filled pixels being 127.

[0075] Afterwards, the model training module can also use image enhancement methods, such as mosaic enhancement and scaling, to enlarge, reduce, or stitch the fourth image, thereby enhancing the diversity of the image.

[0076] After preprocessing, the model training module can determine the annotation information corresponding to each fourth image in the following way: For each fourth image, the second depth information corresponding to the fourth image can be obtained from the depth data acquisition device that has the same acquisition point and frequency as the image acquisition device that acquired the fourth image. The depth data acquisition device can be a TOF (Time of Flight) sensor, etc. For each second object in the fourth image, a rectangular detection box that is parallel and fits the body of the second object is determined, and the coordinate information of the rectangular detection box and the rotation angle information between the long side of the rectangular detection box and the horizontal line are recorded as the second detection box information of the second object.

[0077] When determining the detection box, the model training module can first determine the parallel axis rectangle containing the second object, then rotate the rectangle so that the rotated rectangle fits as closely as possible to the body of the second object, and then record five parameters: the center point coordinates (x, y) of the rotated rectangle, the width w of the rectangle, the height h of the rectangle, and the clockwise rotation angle α around the center point.

[0078] Finally, the model training module uses multiple fourth images as training samples and iteratively trains the initial detection model using these training samples and their annotation information to obtain the target detection model.

[0079] In practical applications, the acquisition module first acquires the first image to be detected, and then the preprocessing module preprocesses the first image in the following way to obtain the second image: the preprocessing module determines the pre-filled area of ​​a preset size around the first image based on the length and width of the first image; the pre-filled area is filled with pixels of a preset pixel value to obtain the second image.

[0080] Then, the detection module can use the pre-trained target detection model to analyze and process the second image to obtain the first detection box information and the first depth information of multiple first objects in the second image.

[0081] Specifically, the detection module can first use the feature extraction module to extract features from the second image to obtain the first feature vector corresponding to the second image; then use the bounding box prediction module to analyze and process the first feature vector to obtain the first detection box information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and the rotation angle information of the detection box relative to the horizontal line. The detection box coordinate information includes: the center point coordinate information of the detection box and the width and height information of the detection box; then use the depth prediction module to analyze and process the first feature vector to obtain the depth image corresponding to the second image, and determine the first depth information of multiple first objects in the second image based on the depth image. The first depth information includes: the first distance between the center point of the detection box and the target image acquisition device, where the target image acquisition device is the image acquisition device that acquires the first image.

[0082] As an optional implementation, when determining the body length of each first object based on the detection box information and depth information, the determining module can first determine the second length of a reference object in the first image whose actual length is the first length, and determine the ratio of the first length to the second length as a mapping parameter. The reference object can be an object with a standard length, such as a fence in the image. Then, for each first object in the second image, the body length of the first object is determined based on the first detection box information, the first depth information, and the mapping parameter corresponding to the first object.

[0083] Optionally, for each first object in the second image, the determining module can determine a second distance between the center point of the reference object and the target image acquisition device based on the depth image, and determine a third length of the longer side of the detection box corresponding to the first object; determine the product of the first distance, the third length and the mapping parameter, and determine the ratio of the product to the second distance as the body length of the first object.

[0084] It should be noted that each module in the target object length detection device in this embodiment corresponds one-to-one with each implementation step of the target object length detection method in Embodiment 1. Since Embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to Embodiment 1, and will not be elaborated further here.

[0085] Example 3

[0086] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device containing the non-volatile storage medium executes the target object length detection method in Embodiment 1 by running the computer program.

[0087] Specifically, the device containing the non-volatile storage medium executes the following steps by running the computer program: acquiring a first image to be detected; preprocessing the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; analyzing and processing the second image using a pre-trained target detection model to obtain first detection box information and first depth information of multiple first objects in the second image, wherein the first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line; determining the body length of each first object based on the first detection box information and the first depth information.

[0088] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the target object length detection method in Embodiment 1.

[0089] Specifically, the computer program executes the following steps during runtime: acquiring a first image to be detected; preprocessing the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; analyzing and processing the second image using a pre-trained object detection model to obtain first detection box information and first depth information of multiple first objects in the second image, wherein the first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line; determining the body length of each first object based on the first detection box information and the first depth information.

[0090] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the target object length detection method of Embodiment 1 through the computer program.

[0091] Specifically, the processor is configured to execute the following steps via a computer program: acquiring a first image to be detected; preprocessing the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; analyzing and processing the second image using a pre-trained object detection model to obtain first detection box information and first depth information of multiple first objects in the second image, wherein the first detection box information includes at least: detection box coordinate information and rotation angle information of the detection box relative to the horizontal line; and determining the body length of each first object based on the first detection box information and the first depth information.

[0092] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0093] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0097] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0098] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting the body length of a target object, characterized in that, include: Obtain the first image to be detected; The first image is preprocessed to obtain the second image, wherein the preprocessing includes at least: image expansion processing; The second image is analyzed and processed using a pre-trained object detection model to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and detection box rotation angle information relative to the horizontal line. The body length of each of the first objects is determined based on the first detection frame information and the first depth information; The training process of the target detection model includes: constructing an initial detection model; acquiring multiple third images captured by image acquisition devices at various acquisition points under various lighting conditions; performing the preprocessing on each third image to obtain multiple fourth images; determining the annotation information corresponding to each fourth image, wherein the annotation information includes at least: the second detection box information of each second object in the fourth image; and iteratively training the initial detection model using the multiple fourth images and the annotation information to obtain the target detection model. The process of determining the annotation information includes: for each second object in the fourth image, determining a rectangular detection box that is parallel to and fits the body of the second object, and recording the coordinate information of the rectangular detection box and the rotation angle information between the long side of the rectangular detection box and the horizontal line as the second detection box information of the second object.

2. The method according to claim 1, characterized in that, The first image is preprocessed to obtain the second image, including: The area to be filled around the first image is determined based on the length and width of the first image; The area to be filled is filled with pixels of a preset pixel value to obtain the second image.

3. The method according to claim 1, characterized in that, The target detection model includes at least: a feature extraction module, a bounding box prediction module, and a depth prediction module. It uses a pre-trained target detection model to analyze and process the second image, obtaining first detection box information and first depth information for multiple first objects in the second image, including: The feature extraction module is used to extract features from the second image to obtain a first feature vector corresponding to the second image; The first feature vector is analyzed and processed using the bounding box prediction module to obtain the first detection box information of multiple first objects in the second image. The detection box coordinate information includes: the center point coordinate information of the detection box and the width and height information of the detection box. The depth prediction module is used to analyze and process the first feature vector to obtain the depth image corresponding to the second image, and the first depth information of multiple first objects in the second image is determined based on the depth image. The first depth information includes: the first distance between the center point of the detection box and the target image acquisition device, wherein the target image acquisition device is the image acquisition device that acquires the first image.

4. The method according to claim 1, characterized in that, in, The annotation information also includes: second depth information corresponding to the fourth image, determining the annotation information corresponding to each fourth image, including: For each of the fourth images, the second depth information corresponding to the fourth image acquired by a depth data acquisition device at the same acquisition point and frequency as the image acquisition device that acquired the fourth image is obtained.

5. The method according to claim 3, characterized in that, Determining the body length of each of the first objects based on the detection frame information and the depth information includes: Determine the second length of the reference object in the first image, whose actual length is the first length, and determine the ratio of the first length to the second length as a mapping parameter; For each of the first objects in the second image, the body length of the first object is determined based on the first detection box information, the first depth information, and the mapping parameters corresponding to the first object.

6. The method according to claim 5, characterized in that, Determining the body length of the first object based on the first detection box information, the first depth information, and the mapping parameters corresponding to the first object includes: Based on the depth image, a second distance is determined between the center point of the reference object and the target image acquisition device, and a third length of the longer side of the detection box corresponding to the first object is determined; The product of the first distance, the third length, and the mapping parameter is determined, and the ratio of the product to the second distance is determined as the body length of the first object.

7. A target object length detection device, characterized in that, include: The acquisition module is used to acquire the first image to be detected; A preprocessing module is used to preprocess the first image to obtain a second image, wherein the preprocessing includes at least: image expansion processing; The detection module is used to analyze and process the second image using a pre-trained target detection model to obtain first detection box information and first depth information of multiple first objects in the second image. The first detection box information includes at least: detection box coordinate information and detection box rotation angle information relative to the horizontal line. The determination module is used to determine the body length of each of the first objects based on the first detection frame information and the first depth information; The training process of the target detection model includes: constructing an initial detection model; acquiring multiple third images captured by image acquisition devices at various acquisition points under various lighting conditions; performing the preprocessing on each third image to obtain multiple fourth images; determining the annotation information corresponding to each fourth image, wherein the annotation information includes at least: the second detection box information of each second object in the fourth image; and iteratively training the initial detection model using the multiple fourth images and the annotation information to obtain the target detection model. The process of determining the annotation information includes: for each second object in the fourth image, determining a rectangular detection box that is parallel to and fits the body of the second object, and recording the coordinate information of the rectangular detection box and the rotation angle information between the long side of the rectangular detection box and the horizontal line as the second detection box information of the second object.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the body length detection method of any one of claims 1 to 6 by running the computer program.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the body length detection method for a target object according to any one of claims 1 to 6 via the computer program.

Images