Measurement method of target object, and device, storage medium and electronic device thereof

Abstract

The application discloses a target object measurement method and device, a storage medium and an electronic device. The measurement method comprises the following steps: acquiring a first color image and a second depth information image based on structured light of a target object; configuring the first color image and the second depth information image in the same reference coordinate system; determining image information of a color image meeting a preset condition in the first color image as image information of a depth information missing image in the second depth information image based on the reference coordinate system, so as to obtain a target image in the reference coordinate system; and performing image processing on the target image to obtain a target size of the target object.

Description

Technical Field

[0001] This application relates to the field of measurement technology, and more specifically, to a method, apparatus, storage medium, and electronic device for measuring a target object. Background Technology

[0002] With the rapid development of logistics warehousing, warehouse management has spread to all walks of life. Warehouse management is an important part of logistics management, providing goods management services such as goods receiving, inventory counting, and goods issuing.

[0003] When goods are received into the warehouse, their information, such as dimensions (length, width, and height), needs to be entered to facilitate comprehensive management and scheduling of the goods.

[0004] Currently, existing technologies generally employ structured light-based measuring devices to measure cargo dimensions. The measurement principle involves the measuring device emitting a light signal that illuminates the surface of the cargo and is reflected. The measuring device then calculates information such as the cargo's position and depth based on changes in the reflected light signal, thereby obtaining the cargo's dimensional information.

[0005] However, the inventors of this application discovered that, due to the limitations of the physical properties of light signals, when the goods are transparent or highly transparent objects (such as mineral water or beverages), the light signals cannot be properly reflected on the surface of the goods, resulting in the inability to measure the size of the goods or inaccurate measurement results. Based on this, the inventors of this application believe that current methods for measuring the size of goods need further improvement. Summary of the Invention

[0006] This application discloses a method, apparatus, storage medium, and electronic device for measuring a target object, which solves the problem that current structured light cannot accurately measure the size of transparent or highly transparent objects.

[0007] According to one aspect of this application, a method for measuring a target object is provided. The method includes acquiring a first color image of the target object and a second depth information image based on structured light; configuring the first color image and the second depth information image in the same reference coordinate system; determining, based on the reference coordinate system, image information of a color image in the first color image that meets preset conditions as image information of an image in the second depth information image that lacks depth information, so as to obtain a target image in the reference coordinate system; and performing image processing on the target image to obtain the target size of the target object.

[0008] According to some embodiments of this application, determining the image information of a color image in a first color image that meets preset conditions as the image information of an image with missing depth information in a second depth information image includes: traversing the depth information of the second depth information image; determining the depth information image with depth information less than a preset threshold as the image with missing depth information; determining the coordinates corresponding to the image with missing depth information; determining the color image in the first color image corresponding to the coordinates based on the coordinate mapping relationship; and determining the color value of the color image as the color value of the image with missing depth information.

[0009] According to some embodiments of this application, image processing of a target image to obtain the target size of a target object includes: traversing the depth information of a second depth information image; determining the depth information image corresponding to depth information greater than a preset threshold; performing background processing on the depth information image to obtain a background image; segmenting the target object image from the target image based on the background image; performing image processing on the target object image to obtain the reference size of the target object in a reference coordinate system; and determining the target size of the target object based on the reference size.

[0010] According to some embodiments of this application, image processing of a target image includes: noise reduction processing of the target image; wherein the noise reduction processing includes at least erosion processing and dilation processing.

[0011] According to another aspect of this application, a measuring device for a target object is provided. The measuring device includes a first image acquisition unit, a second image acquisition unit, and an image processing unit. The first image acquisition unit acquires a first color image of the target object; the second image acquisition unit acquires a second depth information image of the target object based on structured light; the image processing unit arranges the first color image and the second depth information image in the same reference coordinate system, and, based on the reference coordinate system, determines the image information of the color image in the first color image that meets preset conditions as the image information of the image in the second depth information image that lacks depth information, so as to obtain a target image in the reference coordinate system; the image processing unit performs image processing on the target image to obtain the target size of the target object.

[0012] According to some embodiments of this application, the image processing unit traverses the depth information of the second depth information image, determines the depth information image with depth information less than a preset threshold as a depth information missing image, and determines the coordinates corresponding to the depth information missing image; the image processing unit determines the color image in the first color image corresponding to the coordinates based on the coordinate mapping relationship, and determines the color value of the color image as the color value of the depth information missing image.

[0013] According to some embodiments of this application, the image processing unit further traverses the depth information of the second depth information image, determines the depth information image corresponding to the depth information greater than a preset threshold, and performs background processing on the depth information image to obtain a background image; the image processing unit segments the target object image from the target image based on the background image, performs image processing on the target object image to obtain the reference size of the target object in the reference coordinate system, and determines the target size of the target object based on the reference size.

[0014] According to some embodiments of this application, the image processing unit further performs noise reduction processing on the target image; wherein, the noise reduction processing includes at least erosion processing and dilation processing.

[0015] According to another aspect of this application, a non-volatile computer-readable storage medium is also provided. This storage medium stores a computer program that can implement the measurement method described above.

[0016] According to another aspect of this application, an electronic device is also provided. The electronic device includes one or more processors and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the measurement method described above.

[0017] The technical solution of this application configures the first color image and the second depth information image in the same reference coordinate system, and based on the reference coordinate system, determines the image information of the color image in the first color image that meets the preset conditions as the image information of the image in the second depth information image that lacks depth information, so as to obtain the target image in the reference coordinate system, and performs image processing on the target image to obtain the target size of the target object.

[0018] The technical solution provided in this application combines the color image of the target object with the depth information map based on structured light. The missing parts in the depth information map caused by the transparency of the target object are filled in with the color values ​​of the corresponding parts in the color image. This allows the structured light-based measurement method to accurately measure the size (such as length and width) of transparent or highly transparent target objects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1aA front view of a prior art measuring device is shown;

[0021] Figure 1b A side view of a prior art measuring device is shown.

[0022] Figure 2 A schematic flowchart of a measurement method according to an example embodiment of this application is shown;

[0023] Figure 3 A schematic diagram of a measuring device according to an example embodiment of this application is shown;

[0024] Figure 4 A schematic diagram showing a first color image of an example embodiment of this application;

[0025] Figure 5 A schematic diagram showing a second depth information image of an example embodiment of this application;

[0026] Figure 6 Another schematic flowchart illustrating the measurement method of an example embodiment of this application is shown;

[0027] Figure 7 This diagram illustrates yet another flow chart of the measurement method according to an example embodiment of this application;

[0028] Figure 8 A schematic diagram showing a target image of an example embodiment of this application;

[0029] Figure 9 Another schematic diagram showing a target image of an example embodiment of this application;

[0030] Figure 10 Another schematic diagram showing a target image of an example embodiment of this application;

[0031] Figure 11 A schematic diagram of the structure of a measuring device according to an example embodiment of this application is shown.

[0032] Explanation of reference numerals in the attached figures:

[0033] Measuring device 1; base 11; support rod 12; screen 13; gravity detection device 14; distance measuring device 15; camera device 16; control main board 17;

[0034] First image acquisition unit 21; second image acquisition unit 22; image processing unit 23. Detailed Implementation

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0036] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0037] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0038] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order.

[0039] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0040] In the current technology, with the rapid development of Internet technology, online shopping based on e-commerce platforms has become a common new retail model. For example, users can shop online through e-commerce platforms, and the e-commerce platform will dispatch the corresponding goods from offline warehouses according to the user's order information, and complete the offline acquisition of goods through various methods such as goods delivery or user self-pickup.

[0041] Therefore, the management and scheduling of goods in warehousing plays a crucial role in the entire logistics process. When goods first enter the warehouse, their information needs to be entered, such as their dimensions (length, width, and height) and weight, to facilitate subsequent scheduling of storage, handling, and outbound processes.

[0042] For example, in the field of express logistics, it is also necessary to enter information such as the size (length, width, and height) and weight of express packages in order to classify and organize different types of express packages and manage them in a comprehensive manner.

[0043] Currently, existing technologies typically use structured light or RGB image projection to measure the size information of a target object. Figure 1a A front view of a prior art measuring device is shown; Figure 1b A side view of a prior art measuring device is shown. (e.g.) Figure 1a As shown in 1b, the measuring device 1 includes a base 11, a support rod 12, a screen 13, a gravity detection device 14, a distance measuring device 15, a camera device 16, and a control main board 17.

[0044] The gravity detection device 14 is used to acquire the gravity information of the target object, and the ranging device 15 (such as structured light ranging) is used to acquire the size information of the target object (such as length, width, and height). For example, the ranging device 15 projects a laser signal with encoded information onto the surface of the target object, and determines the position and depth information of the target object based on the speckle information of the laser signal reflected back from the surface of the target object.

[0045] The camera device 16 is used to acquire image information of the target object, and the control motherboard 17 determines the size and weight of the target object based on gravity information, size information and image information.

[0046] However, the inventors discovered that the method of measuring the size of a target object using RGB image projection is easily affected by ambient light, resulting in low measurement accuracy. Furthermore, the structured light-based measurement method, due to the limitations of the physical properties of structured light, suffers from limitations when the target object is transparent or highly transparent. In such cases, the light signal is transmitted through the transparent parts of the object instead of being reflected properly, leading to measurement failures or large errors. For example, when the target object is a bottle of water, the measuring device can only detect the reflected signal from the bottle cap, but cannot obtain the reflected signal from the transparent parts of the water. Therefore, the measuring device can only measure the size of the bottle cap, but cannot determine the actual length and width of the water.

[0047] The inventors also discovered that another existing measuring device removes product shadows by adding a backlight and bottom light to reduce the influence of ambient light, and supplements the missing depth information by matching the depth values ​​of adjacent known depth values ​​to points in the depth information map. However, this type of measurement method increases hardware costs, and the accuracy of the supplemented depth values ​​is poor for target objects with a large range of missing depth information.

[0048] Based on this, one aspect of this application provides a method for measuring a target object. This method combines a color image of the target object with a depth information map based on structured light. The missing parts in the depth information map due to the transparency of the target object are filled in with color values ​​using the corresponding part of the image information in the color image. This allows the structured light-based measurement method to accurately measure the size (such as length and width) of transparent or highly transparent target objects.

[0049] The technical solution of this application will now be described in detail with reference to the accompanying drawings.

[0050] Figure 2 A flowchart illustrating a measurement method according to an example embodiment of this application is shown. Figure 2 As shown, the measurement method includes steps S100 to S400. According to the example embodiment, this wiring method is performed by a measuring device.

[0051] In step S100, the measuring device acquires a first color image of the target object and a second depth information image based on structured light.

[0052] According to the example embodiment, the target object refers to the object whose attributes the user wants to measure (such as the size, volume, or weight of the object). For example, the target object can be a certain item in a warehouse, such as merchandise or express delivery, but this application is not limited to this.

[0053] The first color image can be a color image of the target object obtained by the measuring device through the camera device, such as an RGB image; the second depth information image can be a depth information image of the target object obtained by the measuring device through structured light, and the depth information image includes the depth information of each position point of the target object, i.e., the depth value.

[0054] Figure 3 A schematic diagram of a measuring device according to an example embodiment of this application is shown, such as Figure 3 As shown, the measuring device includes a color image camera and a depth information image camera.

[0055] Figure 4 A schematic diagram showing a first color image of an example embodiment of this application; Figure 5 A schematic diagram showing a second depth information image of an example embodiment of this application.

[0056] A color image camera can acquire a first color image of a target object. For example, a color image camera is an RGB camera, which captures an RGB image of the target object (e.g., ...). Figure 4As shown, the target object is a drinking water container; the depth information image camera device can acquire a second depth information image of the target object. For example, the depth information image camera device is an IR camera, which captures an IR image of the target object (such as...). Figure 5 As shown, the target object is a drinking water container.

[0057] This can be understood as the second depth information image being a visualized depth information image. The measuring device visualizes the second depth information image as a grayscale value of 0-255 based on a hash function, to obtain, for example... Figure 5 The grayscale image shown.

[0058] According to the example embodiment, the smaller the depth value, the darker the grayscale value of the corresponding point, and the closer the depth value is to 0. For example... Figure 5 As shown, the depth values ​​of the edge volume of the target object (water drink) are all approximately 0.

[0059] In step S200, the measuring device configures the first color image and the second depth information image in the same reference coordinate system.

[0060] like Figure 3 As shown, due to the different positions of the color image camera and the depth information image camera, and the different built-in parameters of the camera devices, the main fields of view of the first color image and the second depth information image are also different.

[0061] According to the example embodiment, the measuring device establishes a reference coordinate system and aligns the first color image and the second depth information image to the reference coordinate system so that the first color image and the second depth information image have the same coordinate position information.

[0062] The bottom of the target object is equipped with a tray. The vertical height between the depth information image camera of the measuring device and the tray is H, which is the measuring range of the measuring device. For example, H can be 80cm, and the matrix of the reference coordinate system can be 640*480.

[0063] In step S300, the measuring device, based on the reference coordinate system, determines the image information of the color image in the first color image that meets the preset conditions as the image information of the image with missing depth information in the second depth information image, so as to obtain the target image in the reference coordinate system.

[0064] For example, an image in the second depth information image that lacks depth information (depth value of 0 or close to 0) due to the transparency of the target object is considered a depth-incomplete image. The preset condition is that the coordinate position information of the color image in the first color image is the same as that of the depth-incomplete image.

[0065] The measuring device uses the image information of the color image in the first color image as the image information of the image missing depth information in the second depth information image to complete the image information in the second depth information image. After the image information is completed, the measuring device can obtain a target image with complete depth information.

[0066] Figure 6 Another flowchart illustrating the measurement method of an example embodiment of this application is shown.

[0067] Optionally, in step S300, as Figure 6 As shown, the measuring device determines the image information of the color image that meets the preset conditions in the first color image as the image information of the image with missing depth information in the second depth information image, and also includes steps S310 to S350.

[0068] In step S310, the measuring device traverses the depth information of the second depth information image. For example, the measuring device reads the depth information (or depth value) of all points in the second depth information image.

[0069] In step S320, the measuring device determines images with depth information less than a preset threshold as images with missing depth information.

[0070] For example, the preset threshold is the measurement range H of the measuring device. This can be understood as follows: when the measurement range of the measuring device is H, the depth information of the target object is less than H. Therefore, the depth information image with depth information less than the preset threshold is the image of the target object to be measured, that is, the image with missing depth information of the transparent part of the target object.

[0071] In step S330, the measuring device determines the coordinates corresponding to the image with missing depth information.

[0072] In step S340, the measuring device determines the color image corresponding to the coordinates in the first color image based on the coordinate mapping relationship.

[0073] In step S350, the measuring device determines the color value of the color image as the color value of the image lacking depth information.

[0074] According to the example embodiment, the measuring device determines the coordinates of the image with missing depth information in the reference coordinate system, and obtains the color image corresponding to those coordinates in the first color image based on the coordinate mapping relationship, and obtains the color value of that color image. The measuring device then modifies the color value of the image with missing depth information in the second depth information image to match the color value of the color image, thereby supplementing the image information of the image with missing depth information.

[0075] For example, the color values ​​of a color image can be the three-channel color values ​​of an RGB image.

[0076] In step S400, the measuring device performs image processing on the target image to obtain the target size of the target object.

[0077] For example, after obtaining the target image, the measuring device can perform image processing on the target image with complete depth information to obtain the required size of the target object.

[0078] Figure 7 This is yet another schematic flowchart illustrating the measurement method of an example embodiment of this application.

[0079] Optionally, in step S400, as Figure 7 As shown, the image processing of the target image by the measuring device also includes steps S410 to S460.

[0080] In step S410, the measuring device traverses the depth information of the second depth information image.

[0081] For example, the measuring device reads the depth information (or depth value) of all points in the second depth information image.

[0082] In step S420, the measuring device determines the depth information image corresponding to depth information that is greater than a preset threshold.

[0083] For example, the preset threshold is the measurement range H of the measuring device. This can be understood as follows: when the measurement range of the measuring device is H, the depth information of the non-target object is greater than H. Therefore, the depth information image with depth information greater than the preset threshold is the location of the non-target object.

[0084] In step S430, the measuring device performs background processing on the depth information image to obtain a background image.

[0085] Figure 8 A schematic diagram showing a target image of an example embodiment of this application. Figure 9 Another schematic diagram showing a target image of an example embodiment of this application; Figure 10 This is another schematic diagram showing a target image of an example embodiment of this application.

[0086] According to an example embodiment, the measuring device performs color filling processing on depth information images with depth information greater than a preset threshold, for example, filling the depth information image with black to obtain a background image (such as...). Figure 8 (The black background area shown).

[0087] In step S440, the measuring device segments the target object image from the target image based on the background image.

[0088] For example, such as Figure 8As shown, the background-processed target image includes a background image portion and a target object image portion with the texture and color of the target object.

[0089] The measuring device performs image segmentation on the background-processed target image to obtain an image of the target object, such as... Figure 9 As shown.

[0090] In step S450, the measuring device performs image processing on the target object image to obtain the reference dimensions of the target object in the reference coordinate system.

[0091] In step S460, the measuring device determines the target size of the target object based on the reference size.

[0092] For example, such as Figure 10 As shown, the measuring device performs a minimum bounding rectangle calculation on the target object image (e.g., ...). Figure 10 The white checkbox shown in the image is processed to obtain the reference dimensions of the target object in the reference coordinate system, and based on certain conversion relationships, the true dimensions of the target object are determined. For example... Figure 10 As shown, the size of the target object can include at least the length and width of the target object.

[0093] Through the above example embodiments, the technical solution provided by this application combines the color image of the target object and the depth information map based on structured light. The missing part in the depth information map due to the transparency of the target object is filled in with the color value of the corresponding part in the color image. This allows the structured light-based measurement method to accurately measure the size (such as length and width) of transparent or highly transparent target objects.

[0094] Optionally, in step S400, the image processing of the target image by the measuring device may further include noise reduction processing of the target image. Noise reduction processing includes at least erosion processing and dilation processing.

[0095] For example, such as Figure 8 As shown, the background image in the target image after backgrounding includes interference factors with depth information less than a preset threshold. Due to the influence of the cross-section of the tray in the measuring device, there are also interference factors with points having a depth information of 0 in the background image. By performing erosion and dilation processing on the target image after backgrounding, the measuring device can remove interference from the background image, allowing the target object image, including the target object, to be completely segmented, thereby ensuring the accuracy of the measurement.

[0096] One aspect of this application provides a measurement device for a target object. This measurement device combines a color image of the target object with a depth information map based on structured light. The missing parts in the depth information map due to the transparency of the target object are filled in with color values ​​using the corresponding part of the image information in the color image. This allows the structured light-based measurement method to accurately measure the size (such as length and width) of transparent or highly transparent target objects.

[0097] Figure 11 A schematic diagram of the structure of the measuring device in an example embodiment of this application is shown, such as... Figure 11 As shown, the measuring device includes a first image acquisition unit 21, a second image acquisition unit 22, and an image processing unit 23.

[0098] According to the example embodiment, the first image acquisition unit 21 acquires a first color image of the target object, and the second image acquisition unit 22 acquires a second depth information image of the target object based on structured light.

[0099] According to the example embodiment, the target object refers to the object whose attributes the user wants to measure (such as the size, volume, or weight of the object). For example, the target object can be a certain item in a warehouse, such as merchandise or express delivery, but this application is not limited to this.

[0100] The first color image can be the color image of the target object acquired by the first image acquisition unit 21, such as an RGB image; the second depth information image can be the depth information image of the target object acquired by the second image acquisition unit 22 through structured light, and the depth information image includes the depth information of each position point of the target object, i.e., the depth value.

[0101] This can be understood as the second depth information image being a depth information image that has undergone visualization processing. The measuring device visualizes the second depth information image as a grayscale value of 0-255 based on a hash function. According to the example embodiment, the smaller the depth value, the darker the grayscale value of the corresponding point, and the closer the depth value is to 0.

[0102] According to the example embodiment, the image processing unit 23 configures the first color image and the second depth information image in the same reference coordinate system.

[0103] Because the positions of the first image acquisition unit 21 and the built-in parameters of the camera device are different, the main field of view of the first color image and the second depth information image are also different.

[0104] According to the example embodiment, the image processing unit 23 establishes a reference coordinate system and aligns the first color image and the second depth information image to the reference coordinate system so that the first color image and the second depth information image have the same coordinate position information.

[0105] The bottom of the target object is equipped with a tray. The vertical height between the second image acquisition unit 22 of the measuring device and the tray is H, which is the measuring range of the measuring device. For example, H can be 80cm, and the matrix of the reference coordinate system can be 640*480.

[0106] According to the example embodiment, the image processing unit 23 determines the image information of the color image that meets the preset conditions in the first color image as the image information of the image with missing depth information in the second depth information image, so as to obtain the target image in the reference coordinate system.

[0107] The image information of the color image that meets the preset conditions in the first color image is determined as the image information of the image with missing depth information in the second depth information image, so as to obtain the target image in the reference coordinate system;

[0108] The image processing unit 23 uses the image information of the color image in the first color image as the image information of the image missing depth information in the second depth information image to complete the image information in the second depth information image. After the image information is completed, the measuring device can obtain a target image with complete depth information.

[0109] Optionally, the image processing unit 23 traverses the depth information of the second depth information image, determines the depth information image with depth information less than a preset threshold as a depth information missing image, and determines the coordinates corresponding to the depth information missing image.

[0110] For example, the image processing unit 23 reads the depth information (or depth value) of all points in the second depth information image.

[0111] The preset threshold is the measurement range H of the measuring device. This can be understood as follows: when the measurement range of the measuring device is H, the depth information of the target object is less than H. Therefore, the depth information image with depth information less than the preset threshold is the image of the target object to be measured, that is, the image with missing depth information of the transparent part of the target object.

[0112] The image processing unit 23 determines the color image corresponding to the coordinates in the first color image based on the coordinate mapping relationship, and determines the color value of the color image as the color value of the image with missing depth information.

[0113] According to the example embodiment, the image processing unit 23 determines the coordinates of the image with missing depth information in the reference coordinate system, and obtains the color image corresponding to those coordinates in the first color image based on the coordinate mapping relationship, and obtains the color value of that color image. The image processing unit 23 then modifies the color value of the image with missing depth information in the second depth information image to match the color value of the color image, thereby supplementing the image information of the image with missing depth information.

[0114] For example, the color values ​​of a color image can be the three-channel color values ​​of an RGB image.

[0115] According to the example embodiment, the image processing unit 23 performs image processing on the target image to obtain the target size of the target object.

[0116] For example, after obtaining the target image, the image processing unit 23 performs image processing on the target image with complete depth information to obtain the required size of the target object.

[0117] Optionally, the image processing unit 23 further traverses the depth information of the second depth information image, determines the depth information image corresponding to the depth information that is greater than a preset threshold, and performs background processing on the depth information image to obtain a background image.

[0118] For example, the image processing unit 23 reads the depth information (or depth value) of all points in the second depth information image.

[0119] The preset threshold is the measurement range H of the measuring device. This can be understood as follows: when the measurement range of the measuring device is H, the depth information of non-target objects is greater than H. Therefore, depth information images with depth information greater than the preset threshold represent the locations of non-target objects.

[0120] The image processing unit 23 performs color filling processing on depth information images with depth information greater than a preset threshold, for example, filling the depth information image with black to obtain a background image.

[0121] The image processing unit 23 segments the target object image from the target image based on the background image, performs image processing on the target object image to obtain the reference size of the target object in the reference coordinate system, and determines the target size of the target object based on the reference size.

[0122] For example, the background-processed target image includes a background image portion and a target object image portion with the texture and color of the target object. The image processing unit 23 performs image segmentation on the background-processed target image to obtain an image of the target object.

[0123] The image processing unit 23 performs minimum bounding rectangle processing on the target object image to obtain the reference size of the target object in the reference coordinate system, and determines the true size of the target object based on certain conversion relationships. For example, the size of the target object may include at least the length and width of the target object.

[0124] Through the above example embodiments, the technical solution provided by this application combines the color image of the target object and the depth information map based on structured light. The missing part in the depth information map due to the transparency of the target object is filled in with the color value of the corresponding part in the color image. This allows the structured light-based measurement method to accurately measure the size (such as length and width) of transparent or highly transparent target objects.

[0125] Optionally, the image processing unit 23 also performs noise reduction processing on the target image, wherein the noise reduction processing includes at least erosion processing and dilation processing.

[0126] For example, the background image in the target image after backgrounding processing may contain interference factors with depth information less than a preset threshold. Due to the influence of the cross-section of the tray in the measuring device, there may also be interference factors with points having a depth information of 0 in the background image. The image processing unit 23 can remove the interference in the background image by performing erosion and dilation processing on the target image after backgrounding processing, so that the target object image including the target object can be completely segmented in the target image, thereby ensuring the accuracy of the measurement.

[0127] According to another aspect of this application, a non-volatile computer-readable storage medium is also provided. This storage medium stores a computer program that can implement the measurement method described above.

[0128] According to another aspect of this application, an electronic device is also provided. The electronic device includes one or more processors and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the measurement method described above.

[0129] Finally, it should be noted that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions of the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for measuring a target object, characterized in that, include: Acquire a first color image of the target object and a second depth information image based on structured light; The first color image and the second depth information image are configured in the same reference coordinate system; Based on the reference coordinate system, the image information of the color image that meets the preset conditions in the first color image is determined as the image information of the image with missing depth information in the second depth information image, so as to obtain the target image in the reference coordinate system; The target image is processed to obtain the target size of the target object; The step of determining the image information of the color image in the first color image that meets the preset conditions as the image information of the image with missing depth information in the second depth information image includes: Traverse the depth information of the second depth information image; Images with depth information less than a preset threshold are defined as images with missing depth information; the preset threshold is the range of the measuring device; the bottom of the target object is equipped with a tray; and the range is the vertical height between the depth information image camera device and the tray. Determine the coordinates corresponding to the image with missing depth information; Based on the coordinate mapping relationship, the color image in the first color image corresponding to the coordinates is determined; The color values ​​of the color image are determined as the image information of the image lacking depth information; The preset condition is that the coordinate position information of the color image in the first color image is the same as that of the image with missing depth information.

2. The measurement method according to claim 1, characterized in that, The image processing of the target image includes: Traverse the depth information of the second depth information image; Determine the depth information image corresponding to depth information greater than a preset threshold; The depth information image is then subjected to backgrounding processing to obtain a background image; The target object image is segmented from the target image based on the background image; Image processing is performed on the target object image to obtain the reference size of the target object in the reference coordinate system; The target size of the target object is determined based on the reference size.

3. The measurement method according to claim 1, characterized in that, The image processing of the target image includes: The target image is subjected to noise reduction processing; The noise reduction process includes at least corrosion treatment and expansion treatment.

4. A measuring device for a target object, characterized in that, include: The first image acquisition unit acquires the first color image of the target object; The second image acquisition unit acquires a second depth information image of the target object based on structured light. The image processing unit configures the first color image and the second depth information image in the same reference coordinate system, and based on the reference coordinate system, determines the image information of the color image in the first color image that meets the preset conditions as the image information of the image in the second depth information image that lacks depth information, so as to obtain the target image in the reference coordinate system. The image processing unit performs image processing on the target image to obtain the target size of the target object; The image processing unit traverses the depth information of the second depth information image, determines the depth information image with depth information less than a preset threshold as a depth information missing image, and determines the coordinates corresponding to the depth information missing image; the preset threshold is the range of the measuring device, the bottom of the target object is equipped with a tray, and the range is the vertical height between the depth information image camera device and the tray; The image processing unit determines the color image in the first color image corresponding to the coordinates based on the coordinate mapping relationship, and determines the color value of the color image as the image information of the image with missing depth information; The preset condition is that the coordinate position information of the color image in the first color image is the same as that of the image with missing depth information.

5. The measuring device according to claim 4, characterized in that, The image processing unit also traverses the depth information of the second depth information image, determines the depth information image corresponding to the depth information that is greater than a preset threshold, and performs background processing on the depth information image to obtain a background image. The image processing unit segments the target object image from the target image based on the background image, performs image processing on the target object image to obtain the reference size of the target object in the reference coordinate system, and determines the target size of the target object based on the reference size.

6. The measuring device according to claim 4, characterized in that, The image processing unit also performs noise reduction processing on the target image; The noise reduction process includes at least corrosion treatment and expansion treatment.

7. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program implements the measurement method as described in any one of claims 1 to 3.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the measurement method as described in any one of claims 1 to 3.

Images