Object recognition methods, devices, storage media, and computer program products
By using a monocular camera and deep learning technology, the true size of scrap steel can be automatically identified, solving the problems of accuracy and efficiency in manual scrap steel identification. This enables efficient and accurate scrap steel grading, promoting the safe, efficient, and green development of the steel industry.
Patent Information
- Application Number
- CN202410354672.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-26
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-03-26
AI Technical Summary
In existing technologies, manual identification of large-sized scrap steel suffers from low accuracy and low efficiency, and can easily damage equipment, especially during the steel smelting process.
By using a monocular camera combined with deep learning, the system obtains the size of the object to be identified in the image and uses preset conversion coefficients and size thresholds to automatically identify the true size of scrap steel, achieving efficient and accurate size judgment.
It improves the accuracy and efficiency of scrap steel grading, reduces project costs, avoids safety risks in the manual identification process, and enhances the safety and production efficiency of the steel industry.
Smart Images

Figure CN118247557B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to an object recognition method, apparatus, storage medium, and computer program product. Background Technology
[0002] Currently, when scrap steel is recycled and remelted, larger scrap steel can easily damage smelting equipment. Therefore, it is necessary to identify and remove these scrap steels to improve the safety of the production process.
[0003] In general, larger scrap steel is primarily identified manually. This method relies on the operator's experience, resulting in low accuracy and efficiency. Summary of the Invention
[0004] This application provides an object recognition method, apparatus, storage medium, and computer program product, which can efficiently and accurately identify the size of the object to be identified.
[0005] To achieve the above objectives, this application adopts the following technical solution:
[0006] In a first aspect, this application provides an object recognition method, which includes: obtaining the image size of the object to be recognized in a first image; determining the true size of the object to be recognized based on the image size and a preset conversion coefficient; the preset conversion coefficient is established based on a reference true size and a reference image size, wherein the reference true size is a preset true size of the reference object, and the reference image size is a preset size of the reference object in a second image; the camera distance of the first image is the same as the camera distance of the second image; and determining the recognition result of the object to be recognized based on the comparison result between the true size and a preset size threshold.
[0007] In one possible approach, the method further includes: determining the average pixel value of the area occupied by the object to be identified in the depth map of the first image; multiplying the ratio between the average pixel value and the reference average value by the true size to obtain the corrected size of the object to be identified; the reference average value being the average pixel value of the area occupied by a preset reference object in the depth map of the second image; and determining the identification result of the object to be identified based on the comparison result between the corrected size and a preset size threshold.
[0008] In one possible approach, the method further includes: obtaining a reference real size and a second image; performing detection processing on the second image to obtain a reference region occupied by a preset reference object in the second image; determining the diameter of the smallest circumscribed circle corresponding to the reference region as the reference image size; and determining the ratio between the reference real size and the reference image size as a preset conversion coefficient.
[0009] In one possible approach, before obtaining the image size of the object to be identified in the first image, the method further includes: segmenting the first image to obtain multiple object regions; the object region is the area occupied by the object in the first image; performing detection processing on the first image to obtain the region to be identified; the region to be identified is the area in the first image where object identification is required; determining the center coordinates of the minimum circumscribed circle corresponding to each of the multiple object regions; and identifying the object whose center coordinates of the minimum circumscribed circle are located in the region to be identified as the object to be identified.
[0010] In one possible approach, obtaining the image size of the object to be identified in the first image includes: determining the diameter of the smallest circumscribed circle corresponding to the object to be identified as the image size of the object to be identified in the first image.
[0011] In one possible approach, the identification result of the object to be identified is determined based on the comparison result between the actual size and a preset size threshold, including: if the comparison result shows that the actual size is less than the preset size threshold, the size of the object to be identified is determined to be within the normal range as the identification result; if the comparison result shows that the actual size is greater than or equal to the preset size threshold, the size of the object to be identified is determined to be within the abnormal range as the identification result.
[0012] In one possible approach, the method further includes: when the size of the object to be identified is outside the normal range, adding prompt information corresponding to the object to be identified to the first image to obtain a third image; the prompt information is used to indicate that the size of the object to be identified is outside the normal range; and displaying the third image.
[0013] Secondly, this application provides an object recognition device, which includes: an acquisition unit and a determination unit; the acquisition unit is used to acquire the image size of the object to be recognized in a first image; the determination unit is used to determine the true size of the object to be recognized based on the image size and a preset conversion coefficient; the preset conversion coefficient is established based on a reference true size and a reference image size, the reference true size being a preset true size of a reference object, and the reference image size being a preset size of a reference object in a second image; the camera distance of the first image is the same as the camera distance of the second image; the determination unit is further used to determine the recognition result of the object to be recognized based on the comparison result between the true size and a preset size threshold.
[0014] In one possible embodiment, the apparatus further includes: a processing unit; a determining unit, further configured to determine the average pixel value of the area occupied by the object to be identified in the depth map of the first image; a processing unit, configured to multiply the ratio between the average pixel value and the reference average value by the true size to obtain the corrected size of the object to be identified; the reference average value is the average pixel value of the area occupied by a preset reference object in the depth map of the second image; and the determining unit, further configured to determine the identification result of the object to be identified based on the comparison result between the corrected size and a preset size threshold.
[0015] In one possible approach, an acquisition unit is used to acquire a reference real size and a second image; a processing unit is used to perform detection processing on the second image to obtain a reference region occupied by a preset reference object in the second image; a determination unit is further used to determine the diameter of the smallest circumscribed circle corresponding to the reference region as the reference image size; and the determination unit is further used to determine the ratio between the reference real size and the reference image size as a preset conversion coefficient.
[0016] In one possible approach, the acquisition unit is specifically used for: segmenting the first image to obtain multiple object regions; the object region is the region occupied by the object in the first image; performing detection processing on the first image to obtain the region to be identified; the region to be identified is the region in the first image where object identification is required; determining the center coordinates of the minimum circumscribed circle corresponding to each of the multiple object regions; and identifying the object whose center coordinates of the minimum circumscribed circle are located in the region to be identified as the object to be identified.
[0017] In one possible approach, the acquisition unit is specifically used to: acquire the image size of the object to be identified in the first image, including: determining the diameter of the smallest circumscribed circle corresponding to the object to be identified as the image size of the object to be identified in the first image.
[0018] In one possible approach, the determining unit is specifically used to: determine the recognition result of the object to be recognized based on the comparison result between the actual size and a preset size threshold, including: if the comparison result shows that the actual size is less than the preset size threshold, determining the size of the object to be recognized as being within the normal range as the recognition result; if the comparison result shows that the actual size is greater than or equal to the preset size threshold, determining the size of the object to be recognized as being within the abnormal range as the recognition result.
[0019] In one possible embodiment, the device further includes: a display unit; a processing unit, further configured to add prompt information corresponding to the object to be identified to the first image when the size of the object to be identified is outside the normal range, thereby obtaining a third image; the prompt information is used to indicate that the size of the object to be identified is outside the normal range; and a display unit, configured to display the third image.
[0020] Thirdly, this application provides an object recognition device, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the object recognition method as described in the first aspect and any possible implementation of the first aspect.
[0021] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the object recognition method as described in the first aspect and any possible implementation thereof.
[0022] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on an object recognition device, cause the object recognition device to perform the object recognition method as described in the first aspect and any possible implementation thereof.
[0023] In a sixth aspect, embodiments of this application provide a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the object recognition method as described in the first aspect and any possible implementation thereof.
[0024] Specifically, the chip provided in this application embodiment also includes a memory for storing computer programs or instructions.
[0025] In this application, the name of the aforementioned object recognition device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.
[0026] These or other aspects of this application will become more readily apparent in the following description.
[0027] Based on any of the above aspects, the technical solution provided in this application brings at least the following beneficial effects:
[0028] After obtaining the image size of the object to be identified in the first image, the true size of the object to be identified can be determined based on the image size and a preset conversion coefficient established based on the reference true size and the reference image size. Then, the identification result of the object to be identified can be determined based on the comparison result between the true size and the preset size threshold.
[0029] Compared to conventional technologies that rely on manual identification of large-sized scrap steel, this application enables the object recognition device to identify a large number of objects, avoiding the inefficiencies and low accuracy associated with reliance on human experience. Therefore, this application can efficiently and accurately identify the size of the objects to be identified. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the structure of an object recognition system provided in an embodiment of this application;
[0031] Figure 2 This is a schematic diagram of the structure of an object recognition device provided in an embodiment of this application;
[0032] Figure 3 A flowchart illustrating an object recognition method provided in an embodiment of this application;
[0033] Figure 4 A flowchart illustrating yet another object recognition method provided in an embodiment of this application;
[0034] Figure 5 A schematic diagram of a depth map provided in an embodiment of this application;
[0035] Figure 6 A flowchart illustrating yet another object recognition method provided in an embodiment of this application;
[0036] Figure 7 A flowchart illustrating yet another object recognition method provided in an embodiment of this application;
[0037] Figure 8 A schematic diagram of a region to be identified provided in an embodiment of this application;
[0038] Figure 9 A flowchart illustrating yet another object recognition method provided in an embodiment of this application;
[0039] Figure 10 A flowchart illustrating yet another object recognition method provided in an embodiment of this application;
[0040] Figure 11 A schematic diagram of a third image provided for an embodiment of this application;
[0041] Figure 12 A schematic diagram illustrating an object recognition process provided in an embodiment of this application;
[0042] Figure 13 This is a schematic diagram of the structure of an object recognition device provided in an embodiment of this application. Detailed Implementation
[0043] The object recognition method and apparatus provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0044] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0045] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.
[0046] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.
[0047] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0048] The variety, quality, and production cycle of steel production are closely related to the quality and quantity of scrap steel. Therefore, the management and use of scrap steel has become an important focus for steel companies. As the only renewable resource that can replace iron ore in steelmaking, scrap steel has the advantages of being green, environmentally friendly, and recyclable.
[0049] With the acceleration of my country's industrialization, the generation and consumption of scrap steel in steel smelting are also growing rapidly. Therefore, the accurate identification and determination of scrap steel grades has become particularly important, because the quality of scrap steel directly affects the types, quality, and smelting cycle of steel smelting, and thus the economic benefits of steel mills.
[0050] However, scrap steel comes in many varieties, and actual testing scenarios are complex. Traditional scrap steel inspection and grading methods mainly rely on visual inspection, caliper measurement, and joint judgment by supervisors from the production plant and technical center. This method is highly subjective, cumbersome, and prone to disputes regarding grading quality. These problems have long plagued steel companies, becoming bottlenecks restricting the effective utilization of scrap steel and the green development of the steel industry.
[0051] However, with the successful application of deep learning methods in machine vision, the integration of industrial scenarios and deep learning technology is becoming increasingly widespread. This provides a new approach to solving the scrap steel grading problem. By constructing an intelligent scrap steel grading system using deep learning algorithms, automatic identification and accurate grading of scrap steel can be achieved, thereby solving the long-standing scrap steel grading problem in the steel industry.
[0052] Image preprocessing is a crucial step in the scrap steel grading process. Among these steps, the identification of oversized objects is particularly important. If the length of a scrap steel object exceeds the boiler diameter, it can cause molten steel to overflow, affecting production safety and potentially causing severe economic losses to steel companies. Therefore, identifying the length of scrap steel using a monocular camera with centimeter-level accuracy is of great significance for improving inspection efficiency, saving costs, and avoiding safety risks during personnel inspection.
[0053] In conclusion, building an intelligent scrap steel grading system using deep learning technology will help improve the accuracy and efficiency of scrap steel grading and promote the green and sustainable development of steel enterprises.
[0054] Currently, steel mills still primarily rely on manual identification for detecting oversized items in scrap steel. Grading personnel, relying on their extensive experience, make a preliminary judgment on the length of the scrap steel using the naked eye, and then need to climb onto the scrap steel inspection vehicle to manually conduct a detailed assessment. This method is not only inefficient but also poses safety hazards, causing significant inconvenience and risks to the grading personnel.
[0055] To address this issue, we propose an innovative solution: using a monocular camera for detecting excessively long objects. This solution captures images of scrap steel during vehicle unloading using the camera, then utilizes deep learning to generate the outer diameter of each object and uses this diameter to determine if its length exceeds the limit. This method eliminates the need for multiple cameras or expensive sensors such as LiDAR, effectively reducing project costs, while also offering significantly faster detection speeds than 3D cameras or multi-camera systems.
[0056] This solution enables us to achieve centimeter-level object length recognition in a cost-effective and reliable manner, eliminating the need for expensive double-sided cameras. This not only avoids the safety risks associated with manual quality inspection, such as climbing up and down stairs, but also significantly improves inspection efficiency. We believe that the application of this technology will drive the steel industry towards a safer, more efficient, and greener future.
[0057] To address the aforementioned technical deficiencies, the object recognition method provided in this application, after obtaining the image size of the object to be recognized in the first image, can determine the true size of the object to be recognized based on the image size and a preset conversion coefficient established based on the reference true size and the reference image size, so as to further determine the recognition result of the object to be recognized based on the comparison result between the true size and the preset size threshold.
[0058] Compared to conventional technologies that rely on manual identification of large-sized scrap steel, this application enables the object recognition device to identify a large number of objects, avoiding the low efficiency and accuracy issues caused by dependence on human experience. Therefore, this application can efficiently and accurately identify the size of the objects to be identified.
[0059] For example, such as Figure 1 The diagram shown is a structural schematic of an object recognition system 100 provided in an embodiment of this application. The object recognition system 100 may include an object recognition device 101 and a data acquisition device 102. A communication connection may be established between the object recognition device 101 and the data acquisition device 102.
[0060] In practical applications, the object recognition device 101 can communicate with one or more data acquisition devices 102.
[0061] For ease of understanding, this application uses the communication connection between an object recognition device 101 and a data acquisition device 102 as an example for illustration.
[0062] Optionally, Figure 1 The object recognition device 101 and the data acquisition device 102 can be functional modules integrated into the same device, or they can be two independently set devices. This application does not impose any restrictions on this.
[0063] It is easy to understand that when the object recognition device 101 and the data acquisition device 102 are functional modules integrated within the same device, the communication method between the object recognition device 101 and the data acquisition device 102 is the same as the communication method between internal modules of the device. In this case, the communication process between the two is the same as that between the object recognition device 101 and the data acquisition device 102 when they are set up independently.
[0064] For ease of understanding, this application mainly uses the example of the object recognition device 101 and the data acquisition device 102 being configured independently of each other.
[0065] Figure 1The object recognition device 101 can be configured with preset conversion coefficients and preset size thresholds. The object recognition device 101 can send an acquisition command to the data acquisition device 102 to acquire a first image. The data acquisition device 102 can send the acquired first image to the object recognition device 101. The object recognition device 101 can acquire the image size of the object to be recognized in the first image. The object recognition device 101 can determine the true size of the object to be recognized by using the image size of the object in the first image and the preset conversion coefficients, and determine the recognition result of the object to be recognized based on the comparison result between the true size and the preset size threshold.
[0066] Optionally, Figure 1 The object recognition device 101 and data acquisition device 102 can be terminals, servers, or other types of electronic devices. Figure 1 The diagram shown is merely an example of the device configuration of the object recognition device 101 and the data acquisition device 102, and does not constitute a limitation thereof.
[0067] When the object identification device 101 and the data acquisition device 102 are terminals, the terminal can be a device providing voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The terminal can communicate with one or more core networks via a radio access network (RAN). The terminal can be a mobile terminal, such as a computer with a mobile terminal, or a portable, pocket-sized, handheld, or computer-embedded mobile device that exchanges voice and / or data with the radio access network, such as a mobile phone, tablet computer, laptop computer, netbook, or personal digital assistant (PDA). This application does not impose any limitations on this.
[0068] When the object identification device 101 and the data acquisition device 102 are servers, the server can be a single server or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This application does not impose any limitations in this regard.
[0069] Optionally, the data acquisition device 102 can be a camera device. For example, the data acquisition device can be a monocular camera or a binocular camera. This application does not impose specific limitations in this regard.
[0070] like Figure 2The diagram shown is a hardware structure schematic of an object recognition device provided in an embodiment of this application. The object recognition device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, memory 22, and communication interface 23 are connected via the bus 24.
[0071] Processor 21 is the control center of the object recognition device. It can be a single processor or a collective term for multiple processing elements. For example, processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.
[0072] As one embodiment, processor 21 may include one or more CPUs, for example Figure 2 CPU 0 and CPU 1 are shown in the diagram.
[0073] The memory 22 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.
[0074] In one possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 via a bus 24 and is used to store instructions or program code. When the processor 21 calls and executes the instructions or program code stored in the memory 22, it can implement the object recognition method provided in the following embodiments of this application.
[0075] In another possible implementation, the memory 22 can also be integrated with the processor 21.
[0076] Communication interface 23 is used for the object identification device to connect with other devices via a communication network, which can be Ethernet, wireless access network, wireless local area network (WLAN), etc. Communication interface 23 may include a receiving unit for receiving data and a transmitting unit for sending data.
[0077] Bus 24 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0078] It should be pointed out that, Figure 2 The structure shown does not constitute a limitation on the object recognition device, except Figure 2 In addition to the components shown, the object recognition device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.
[0079] like Figure 3 The diagram shown is a flowchart illustrating an object recognition method provided in an embodiment of this application. This object recognition method can be applied to... Figure 1 The object recognition device 101 of the object recognition system 100 shown. The object recognition method includes: S301-S303.
[0080] S301, The object recognition device acquires the image size of the object to be recognized in the first image.
[0081] Optionally, Figure 1 The data acquisition device in the object recognition system 100 shown can be a camera device. This application does not impose specific limitations on this.
[0082] In one possible approach, the camera device can capture an image of the area where the object to be identified is located, obtaining a first image. The camera device can then send the first image to the object recognition device. The object recognition device can then obtain the image dimensions of the object to be identified within the first image.
[0083] In one possible approach, the object recognition device can determine the image size of the object in the first image as the diameter of the smallest circumscribed circle corresponding to the object to be recognized. The specific implementation of the object recognition device obtaining the image size of the object in the first image can be found in S701 below. It will not be elaborated further here.
[0084] Optionally, the object to be identified can be set according to actual needs. For example, the object to be identified can be scrap steel or iron ore. This application does not impose specific restrictions in this regard.
[0085] S302. The object recognition device determines the true size of the object to be recognized based on the image size and a preset conversion coefficient.
[0086] The preset conversion coefficient can be established based on the reference real size and the reference image size. The reference real size can be the preset real size of the reference object, and the reference image size can be the preset size of the reference object in the second image. The camera distance of the first image can be the same as the camera distance of the second image.
[0087] In one possible approach, the object recognition device can acquire a reference real size and a second image. The object recognition device can perform detection processing on the second image to obtain a reference region occupied by a preset reference object in the second image. The object recognition device can determine the diameter of the smallest circumscribed circle corresponding to the reference region as the reference image size. The object recognition device can determine the ratio between the reference real size and the reference image size as a preset conversion coefficient. The specific implementation of the object recognition device determining the preset conversion coefficient can be found in S501-S504 below. It will not be elaborated further here.
[0088] In one possible approach, the second image can be the first image. Camera distance can be used to represent the distance between the camera device and the object to be identified.
[0089] Optionally, the preset reference object can be set according to actual needs. For example, the preset reference target can be a square with a real side length of 5 centimeters or a circle with a real diameter of 6 centimeters. This application does not impose specific limitations in this regard.
[0090] In one possible approach, the object recognition device can determine the true size of the object to be recognized by multiplying the image size by a preset conversion coefficient.
[0091] S303. The object recognition device determines the recognition result of the object to be recognized based on the comparison result between the actual size and the preset size threshold.
[0092] Optionally, the preset size threshold can be set according to actual needs. For example, the preset size threshold can be one meter or 80 centimeters. This application does not impose specific limitations in this regard.
[0093] Specifically, the object recognition device can determine the size of the object to be recognized as being within the normal range if the comparison result shows that the actual size is less than a preset size threshold.
[0094] Alternatively, if the comparison result shows that the actual size is greater than or equal to a preset size threshold, the object recognition device may determine that the size of the object to be recognized is outside the normal range as the recognition result.
[0095] Based on the above technical solution, the object recognition method provided in this application, after obtaining the image size of the object to be recognized in the first image, can determine the real size of the object to be recognized based on the image size and a preset conversion coefficient established based on the reference real size and the reference image size, so as to further determine the recognition result of the object to be recognized based on the comparison result between the real size and the preset size threshold.
[0096] Compared to conventional technologies that rely on manual identification of large-sized scrap steel, this application enables the object recognition device to identify a large number of objects, avoiding the low efficiency and accuracy issues caused by dependence on human experience. Therefore, this application can efficiently and accurately identify the size of the objects to be identified.
[0097] In one embodiment, such as Figure 4 As shown, the object recognition method provided in this application embodiment also includes: S401-S403.
[0098] S401, The object recognition device determines the average pixel value of the area occupied by the object to be recognized in the depth map of the first image.
[0099] Understandably, a depth map can be a grayscale image, where each pixel represents the actual distance from the camera device.
[0100] For example, such as Figure 5 The image shown is a schematic diagram of a depth map. Different colors in the depth map represent different distances from the camera device.
[0101] Specifically, the object recognition device can determine the area occupied by the object to be recognized in the depth map of the first image, as well as the number of pixels covered by the occupied area and the value of each pixel. Based on the number of pixels and the value of each pixel, the object recognition device can determine the average pixel value of the area occupied by the object in the depth map of the first image.
[0102] In one possible approach, the object recognition device can determine the pixel coordinates of a preset reference object in the second image. The object recognition device can determine the minimum and maximum x-coordinates, as well as the maximum and minimum y-coordinates, to obtain a minimum matrix covering the preset reference object. The object recognition device can determine the area occupied by the minimum matrix covering the preset reference object in the depth map of the first image, the number of pixels covered by the occupied area, and the value of each pixel. Based on the number of pixels and the value of each pixel, the object recognition device can determine the average pixel value of the area occupied by the object to be recognized in the depth map of the first image.
[0103] For example, the object recognition device can determine a mask image of the object to be recognized. The object recognition device can determine the abscissa with the smallest value (mask min x) and the abscissa with the largest value.
[0104] The minimum matrix covering the preset reference object is obtained by taking the maximum ordinate (mask max x), the maximum ordinate (mask max y), and the minimum ordinate (mask miny). The object recognition device can determine the area occupied by the minimum matrix covering the preset reference object in the depth map of the first image, as well as the number of pixels covered by the occupied area and the value of each pixel. Based on the number of pixels and the value of each pixel, the object recognition device can determine the average pixel value of the area occupied by the object to be recognized in the depth map of the first image.
[0105] S402, The object recognition device multiplies the ratio between the pixel mean and the reference mean by the actual size to obtain the corrected size of the object to be recognized.
[0106] The reference mean can be the average pixel value of the area occupied by the preset reference object in the depth map of the second image.
[0107] Specifically, the object recognition device can determine the area occupied by a preset reference object in the depth map of the second image, as well as the number of pixels covered by that area and the value of each pixel. Based on the number of pixels and the value of each pixel, the object recognition device can determine the average pixel value of the area occupied by the preset reference object in the depth map of the second image, and define this average pixel value as the reference average. The object recognition device can then multiply the ratio between the average pixel value and the reference average by the actual size to obtain the corrected size of the object to be recognized.
[0108] S403. The object recognition device determines the recognition result of the object to be recognized based on the comparison result between the corrected size and the preset size threshold.
[0109] In one possible approach, the object recognition device multiplies the ratio between the pixel mean and the reference mean by the actual size to obtain the corrected size of the object to be recognized. The specific implementation of this method can be found in S303 above. Further details are omitted here.
[0110] Based on this, the object recognition method provided in this application corrects the true size by using the average pixel value of the object to be recognized and the average pixel value of a preset reference object, thereby making the determined true size of the object to be recognized more accurate. Therefore, this application can accurately recognize the size of the object to be recognized.
[0111] In one embodiment, such as Figure 6As shown, the object recognition method provided in this application embodiment further includes: S501-S504.
[0112] S501, The object recognition device acquires a reference real size and a second image.
[0113] It is understandable that the reference real size can be used to characterize the real size of the preset reference object.
[0114] In one possible approach, the object recognition device may be equipped with a storage module. The storage module may store reference dimensions configured by the operator. The object recognition device can read the reference dimensions from the storage module.
[0115] In one possible approach, the specific implementation of the object recognition device acquiring the second image can be referred to the implementation of the object recognition device acquiring the first image in S301, which will not be repeated here.
[0116] S502, The object recognition device performs detection processing on the second image to obtain the reference area occupied by the preset reference object in the second image.
[0117] In one possible approach, the object recognition device can detect the second image, determine the reference area occupied by the preset reference object in the second image, and mark the reference area occupied by the preset reference object on the second image.
[0118] For example, the preset reference target can be a square with a side length of 5 centimeters. The object recognition device performs detection processing on the second image to determine the location of the square, that is, the reference area occupied by the square in the second image, and marks the reference area occupied by the square on the second image.
[0119] S503, The object recognition device determines the diameter of the smallest circumscribed circle corresponding to the reference area as the reference image size.
[0120] Understandably, the smallest circumcircle can be used to represent the smallest circle that completely encloses an object. The distance between the two farthest points on an object is the diameter of this smallest circumcircle.
[0121] Specifically, the object recognition device can determine the minimum circumcircle corresponding to the reference area and determine the diameter of the minimum circumcircle, so as to determine the diameter of the minimum circumcircle corresponding to the reference area as the reference image size.
[0122] Alternatively, the object recognition device can determine the size of the reference image in other ways. For example, the object recognition device can determine the spatial partitioning algorithm corresponding to the reference region using a spatial partitioning algorithm. This application does not impose specific limitations in this regard.
[0123] S504. The object recognition device determines the preset conversion coefficient by the ratio between the actual size and the size of the reference image.
[0124] In one possible approach, the object recognition device can determine the ratio between the reference real size and the reference image size as a preset conversion coefficient after determining the reference real size and the image size.
[0125] For example, the reference real size can be 5 cm, the reference image size can be 5 mm, and the object recognition device can determine that the ratio between the reference real size and the reference image size is 10. The object recognition device can determine 10 as a preset conversion coefficient.
[0126] Based on this, this application can determine the reference real size and reference image size of the preset reference object, and determine preset conversion coefficients based on the reference real size and reference image size, so as to support the efficient and accurate determination of the size of the object to be identified and the recognition result in subsequent processes according to the preset conversion coefficients. Therefore, this application can efficiently and accurately identify the size of the object to be identified.
[0127] In one embodiment, such as Figure 7 As shown, before obtaining the image size of the object to be identified in the first image, the object recognition method provided in this application also includes S601-S604.
[0128] S601. The object recognition device performs segmentation processing on the first image to obtain multiple object regions.
[0129] The object region is the area occupied by the object in the first image.
[0130] Specifically, the object recognition device can identify multiple objects in a first image and segment the first image according to the contours of the multiple objects to obtain multiple object regions.
[0131] Optionally, the method for segmenting the first image can be set according to actual needs. For example, threshold segmentation methods and edge segmentation techniques can be used. This application does not impose specific limitations in this regard.
[0132] S602, The object recognition device performs detection processing on the first image to obtain the area to be recognized.
[0133] The region to be identified is the area in the first image where object recognition is required.
[0134] Optionally, the area to be identified can be set according to actual needs. For example, the area to be identified can be inside the truck bed or other areas. This application does not impose specific restrictions in this regard.
[0135] Optionally, the region recognition algorithm used by the object recognition device to detect and process the first image can be set according to actual needs. For example, the object recognition device can detect the first image according to the target detection (you only look once, YOLO) algorithm. This application does not impose specific limitations in this regard.
[0136] For example, such as Figure 8 The image shown is a schematic diagram of a region to be identified. Figure 8 It includes multiple objects to be identified.
[0137] For example, the object recognition device can detect the first image according to the YOLO algorithm, determine the region to be recognized, and convert the region to be recognized into a mask image.
[0138] S603, The object recognition device determines the center coordinates of the smallest circumcircle corresponding to each of the multiple object regions.
[0139] It is understandable that the specific implementation method by which the object recognition device determines the minimum circumscribed circle corresponding to each of the multiple object regions is the same as the method used by the object recognition device in S503. This will not be elaborated further here.
[0140] In one possible approach, the object recognition device can determine the coordinates of each pixel in the first image. The object recognition device can also determine the center coordinates of the smallest circumcircle corresponding to each of multiple object regions.
[0141] For example, the object recognition device can determine the center coordinates (x, y) of the smallest circumcircle corresponding to each of multiple object regions.
[0142] S604. The object recognition device identifies objects whose center coordinates are located in the region to be recognized as objects to be recognized.
[0143] Specifically, the object recognition device can determine the coordinates of the area to be recognized and whether the center coordinates of the smallest circumscribed circle are within the coordinates of the area to be recognized. The object recognition device can identify objects whose center coordinates of the smallest circumscribed circle are within the area to be recognized as objects to be recognized.
[0144] For example, when the area to be identified is the area inside the carriage, the object recognition device can identify the object whose center coordinates of the smallest circumscribed circle are located in the area inside the carriage as the object to be identified.
[0145] Based on this, this application can determine whether an object is the object to be identified based on the midpoint coordinates of each object in the first image before obtaining the image size of the object to be identified in the first image. This application can efficiently and accurately identify the size of the object to be identified.
[0146] In one embodiment, combined with Figure 3 When obtaining the image size of the object to be identified in the first image, such as Figure 9 As shown, the object recognition method provided in this application also includes S701.
[0147] S701. The object recognition device determines the diameter of the smallest circumscribed circle corresponding to the object to be recognized as the image size of the object to be recognized in the first image.
[0148] Understandably, the smallest circumcircle can be used to represent the smallest circle that completely encloses an object. The distance between the two farthest points on an object is the diameter of this smallest circumcircle.
[0149] Specifically, the object recognition device can determine the region corresponding to the object to be recognized. The object recognition device can construct the smallest circumcircle of the region corresponding to the object to be recognized and determine the diameter of the smallest circumcircle, so as to determine the diameter of the smallest circumcircle corresponding to the reference region as the reference image size.
[0150] Based on this, the present application can accurately determine the image size of the object to be identified in the first image, so as to support the object recognition device in accurately determining the true size of the object to be identified.
[0151] In one embodiment, such as Figure 10 As shown, the object recognition method provided in this application also includes S801-S802.
[0152] S801. When the size of the object to be identified is outside the normal range, the object recognition device adds prompt information corresponding to the object to be identified to the first image to obtain a third image.
[0153] The prompt message is used to indicate that the size of the object to be identified is outside the normal range.
[0154] In one possible approach, the object recognition device can determine whether the size of the object to be recognized is within an abnormal range. If the size of the object to be recognized is within an abnormal range, the object recognition device can add prompt information corresponding to the object to be recognized to the first image, thus obtaining a third image.
[0155] Alternatively, if the size of the object to be identified is within a normal range, the object recognition device can add prompt information corresponding to the object to be identified to the first image to obtain a fourth image.
[0156] S802, The object recognition device displays a third image.
[0157] In one possible configuration, the object recognition device may also be equipped with a display module. The display module can display a third image to prompt the operator regarding the recognition result of the first image.
[0158] For example, combined Figure 8 ,like Figure 11 The image shown is a schematic diagram of a third image provided in this application, including multiple objects in an abnormal range.
[0159] Based on this, this application can annotate objects in an image whose size is outside the normal range, thereby displaying the recognition results and improving detection efficiency.
[0160] In a real-time example, such as Figure 12 The diagram shown is a schematic of an object recognition process provided in this application.
[0161] In one possible approach, the object recognition device can determine a preset reference object. The object recognition device can acquire a first image and a corresponding depth map. The object recognition device can determine the reference real size and reference image size of the preset reference object, as well as the reference region it occupies in the first image. The object recognition device can determine the ratio between the reference real size and the reference image size as a preset transformation coefficient, and determine the reference mean value corresponding to the reference region. The object recognition device can determine the region occupied by the object to be recognized in the first image. The object recognition device can determine the pixel mean value corresponding to the region occupied by the object to be recognized. The object recognition device can determine the ratio between the pixel mean value and the reference mean value as a correction coefficient. The object recognition device can determine multiple objects to be recognized in the first image. The object recognition device can determine the image size and center coordinates of each object to be recognized. The object recognition device can obtain the real size of the object to be recognized based on the preset transformation coefficient, correction coefficient, and real size. The object recognition device can perform detection processing on the first image using a preset algorithm to obtain the region to be recognized. The object recognition device can perform transformation processing on the region to be recognized to obtain a mask map of the region to be recognized. An object recognition device can determine whether an object to be recognized is within a recognition area. The device can determine the recognition result based on a comparison between the actual size and a preset size threshold.
[0162] Optionally, the object recognition device can determine the true length of the object to be recognized according to the following first formula.
[0163]
[0164] Where R can be used to represent the actual size of the object to be identified. d can be used to represent the image size of the object to be identified. m can be used to represent the preset transformation coefficient. x can be used to represent the pixel mean of the object to be identified. y can be used to represent the reference mean of the preset reference object.
[0165] For example, such as Figure 13The diagram shown is a structural schematic of an object recognition device provided in an embodiment of this application. The object recognition device is used to execute the object recognition method provided in this application. For example, the above... Figure 3 , Figure 4 , Figure 6 , Figure 7 , Figure 9 and Figure 10 The object recognition method in the image. The object recognition device may include: an acquisition unit 901, a determination unit 902, a processing unit 903, and a display unit 904.
[0166] In one possible approach, an acquisition unit is used to acquire the image size of the object to be identified in the first image. For example, acquisition unit 901 is used to perform... Figure 3 Step S301 in the process.
[0167] In one possible approach, the determining unit 902 is used to determine the true size of the object to be identified based on the image size and a preset conversion coefficient. For example, the determining unit 902 is used to perform... Figure 3 Step S302 in the process.
[0168] In one possible approach, the determining unit 902 is further configured to determine the recognition result of the object to be recognized based on a comparison between the actual size and a preset size threshold. For example, the determining unit 902 is further configured to perform... Figure 3 Step S303 in the process.
[0169] In one possible approach, the determining unit 902 is further configured to determine the average pixel value of the region occupied by the object to be identified in the depth map of the first image. For example, the determining unit 902 is further configured to perform... Figure 4 Step S401 in the process.
[0170] In one possible approach, processing unit 903 is configured to multiply the ratio between the pixel mean and the reference mean by the true size to obtain the corrected size of the object to be identified. For example, processing unit 903 is configured to perform... Figure 4 Step S402 in the process.
[0171] In one possible approach, the determining unit 902 is further configured to determine the recognition result of the object to be recognized based on a comparison between the corrected size and a preset size threshold. For example, the determining unit 902 is further configured to perform... Figure 4 Step S403 in the process.
[0172] In one possible approach, the acquisition unit 901 is further configured to acquire the reference real size and the second image. For example, the acquisition unit 901 is also configured to perform... Figure 6 Step S501 in the process.
[0173] In one possible approach, the processing unit 903 is further configured to perform detection processing on the second image to obtain a reference region occupied by a preset reference object in the second image. For example, the processing unit 903 is further configured to perform... Figure 6 Step S502 in the process.
[0174] In one possible approach, the determining unit 902 is further configured to determine the diameter of the smallest circumcircle corresponding to the reference region as the reference image size. For example, the determining unit 902 is further configured to perform... Figure 6 Step S503 in the process.
[0175] In one possible approach, the determining unit 902 is further configured to determine the ratio between the reference real size and the reference image size as a preset conversion coefficient. For example, the determining unit 902 is further configured to perform... Figure 6 Step S504 in the process.
[0176] In one possible approach, the acquisition unit 901 is specifically configured to: segment the first image to obtain multiple object regions; each object region is the region occupied by an object in the first image; perform detection processing on the first image to obtain a region to be identified; each region to be identified is the region in the first image where object identification is required; determine the center coordinates of the smallest circumscribed circle corresponding to each of the multiple object regions; and identify objects whose center coordinates of the smallest circumscribed circle are located within the region to be identified as objects to be identified. For example, the acquisition unit 901 is also configured to perform... Figure 7 Steps S601-S604 in the process.
[0177] In one possible approach, the acquisition unit 901 is specifically used to: acquire the image size of the object to be identified in the first image, including: determining the diameter of the smallest circumscribed circle corresponding to the object to be identified as the image size of the object to be identified in the first image. For example, the determination unit 902 is further used to perform... Figure 9 Step S701 in the process.
[0178] In one possible approach, the determining unit 902 is specifically used to: determine the size of the object to be identified as being within the normal range as the identification result when the comparison result shows that the actual size is less than a preset size threshold; and determine the size of the object to be identified as being within the abnormal range as the identification result when the comparison result shows that the actual size is greater than or equal to the preset size threshold.
[0179] In one possible approach, processing unit 903 is further configured to add prompt information corresponding to the object to be identified to the first image when the size of the object to be identified is outside the normal range, thereby obtaining a third image. For example, processing unit 903 is further configured to perform... Figure 10 Step S801 in the process.
[0180] In one possible configuration, display unit 904 is used to display a third image. For example, display unit 904 is also used to perform... Figure 10 Step S802 in the process.
[0181] also, Figure 13 The technical effects of the object recognition device can be referred to the technical effects of the object recognition method in the above embodiments, and will not be repeated here.
[0182] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0183] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the object recognition method described in the above method embodiments.
[0184] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the object identification method in the method flow shown in the above method embodiments.
[0185] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0186] Since the object recognition device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above method, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.
[0187] The above is merely a specific implementation method of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An object recognition method, characterized in that, include: Obtain the image dimensions of the object to be identified in the first image; The true size of the object to be identified is determined based on the image size and the preset conversion coefficient. The preset conversion coefficient is established based on the reference real size and the reference image size. The reference real size is the actual size of the preset reference object, and the reference image size is the size of the preset reference object in the second image. The camera distance of the first image and the camera distance of the second image are the same. Determine the average pixel value of the region occupied by the object to be identified in the depth map of the first image; The corrected size of the object to be identified is obtained by multiplying the ratio between the average pixel value and the reference average value by the true size. The reference mean is the average pixel value of the area occupied by the preset reference object in the depth map of the second image; The identification result of the object to be identified is determined based on the comparison between the corrected size and the preset size threshold.
2. The method according to claim 1, characterized in that, The method further includes: Obtain the reference true size and the second image; The second image is processed by detection to obtain the reference area occupied by the preset reference object in the second image; The diameter of the smallest circumcircle corresponding to the reference region is determined as the size of the reference image; The ratio between the reference real size and the reference image size is determined as the preset conversion coefficient.
3. The method according to claim 1, characterized in that, Before obtaining the image size of the object to be identified in the first image, the method further includes: The first image is segmented to obtain multiple object regions; each object region is the area occupied by an object in the first image. The first image is subjected to detection processing to obtain the region to be identified; the region to be identified is the area in the first image where object recognition is required. Determine the center coordinates of the minimum circumcircle corresponding to each of the multiple object regions; The object whose center coordinates lie within the region to be identified is defined as the object to be identified.
4. The method according to claim 3, characterized in that, The step of obtaining the image size of the object to be identified in the first image includes: The diameter of the smallest circumcircle corresponding to the object to be identified is determined as the image size of the object to be identified in the first image.
5. The method according to claim 1, characterized in that, The step of determining the recognition result of the object to be recognized based on the comparison result between the corrected size and the preset size threshold includes: If the comparison result shows that the corrected size is less than the preset size threshold, the size of the object to be identified is determined to be within the normal range as the identification result; If the comparison result shows that the corrected size is greater than or equal to the preset size threshold, the size of the object to be identified is determined to be in an abnormal range as the identification result.
6. The method according to claim 4, characterized in that, The method further includes: If the size of the object to be identified is outside the normal range, a prompt message corresponding to the object to be identified is added to the first image to obtain a third image; the prompt message is used to indicate that the size of the object to be identified is outside the normal range. The third image is displayed.
7. An object recognition device, characterized in that, include: Acquisition unit, determination unit, and processing unit; The acquisition unit is used to acquire the image size of the object to be identified in the first image; The determining unit is used to determine the true size of the object to be identified based on the image size and a preset conversion coefficient; The preset conversion coefficient is established based on the reference real size and the reference image size. The reference real size is the actual size of the preset reference object, and the reference image size is the size of the preset reference object in the second image. The camera distance of the first image and the camera distance of the second image are the same. The determining unit is used to determine the average pixel value of the area occupied by the object to be identified in the depth map of the first image; The processing unit is further configured to multiply the ratio between the pixel mean and the reference mean by the true size to obtain the corrected size of the object to be identified; the reference mean is the pixel mean of the area occupied by the preset reference object in the depth map of the second image; The determining unit is further configured to determine the recognition result of the object to be identified based on the comparison result between the corrected size and the preset size threshold.
8. An object recognition device, characterized in that, include: A processor and a communication interface; the communication interface is coupled to the processor, the processor being used to run computer programs or instructions to implement the object recognition method as described in any one of claims 1-6.
9. A computer-readable storage medium storing instructions, characterized in that, When the computer executes the instruction, the computer performs the object recognition method as described in any one of claims 1-6.
10. A computer program product containing instructions, characterized in that, When the instructions are executed by a computer, the computer performs the object recognition method as described in any one of claims 1-6.
Citation Information
Patent Citations
Visual positioning sorting method
CN103706571A
Dimension estimation apparatus, dimension estimation method, program, and recording medium
JP2024022922A
