Object area detection method, detection device, detection system and storage medium
By using a large-scale deep learning model to train the training data, generate a target large model, and use the model to process the detected images, the problem that traditional methods are difficult to distinguish overlapping objects is solved, and higher detection accuracy and stability are achieved.
Patent Information
- Application Number
- CN202411995796.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-25
AI Technical Summary
Traditional object area detection algorithms are difficult to distinguish overlapping objects, resulting in the inability to accurately detect objects in complex scenarios.
By using a large-scale deep learning model to train the training data, generate a target big model, and use the model to process the detected images to achieve object area detection.
Large models have stronger recognition capabilities and stability, and can accurately detect objects in complex scenarios, improve detection accuracy and stability, and make up for the shortcomings of traditional methods.
Smart Images
Figure CN119888185A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of industrial detection technology, and in particular to an object region detection method, an object region detection device, an object region detection system, and a computer-readable storage medium. Background Art
[0002] There are currently many application scenarios that require the size of objects to be distinguished and screened, such as the sand and gravel aggregate screening scenario in the construction industry. Modern screening processes use aggregate screens to separate materials according to size. Different aggregate screens have different mesh sizes, which can screen sand and gravel aggregates of different size categories. Therefore, the sand and gravel aggregates that pass through the aggregate screen have a clear upper limit for the area (aggregate diameter) of each sand and gravel aggregate area. When sand and gravel aggregates exceed the upper limit, it will directly affect the quality of this batch of stones, and this phenomenon usually also means that the aggregate screen is damaged to a certain extent. Therefore, the detection of larger sand and gravel aggregates plays a vital role in the screening process of sand and gravel aggregates. In related technologies, image processing can be used to achieve regional detection of stones. However, for the scene of regional detection of irregular objects such as stones, when there is overlap between stones, the traditional regional detection algorithm cannot distinguish the contours of the overlapping stones, so it is difficult to achieve regional analysis of a single stone. Summary of the invention
[0003] The embodiments of the present application provide an object region detection method, an object region detection device, an object region detection system and a computer-readable storage medium to solve at least one of the above-mentioned technical problems.
[0004] The object region detection method provided in the embodiment of the present application includes:
[0005] Acquire training data, the training data including historical images containing the object and historical region annotation results of the object corresponding to the historical images;
[0006] Using the training data to train a pre-selected large model to obtain a target large model; acquiring an image to be detected containing the object;
[0007] The target large model is used to process the image to be detected to obtain a target area detection result of the object in the image to be detected.
[0008] In some embodiments, obtaining training data includes:
[0009] Acquire a plurality of the historical images, where the plurality of the historical images are images acquired by capturing images of the object in various shapes and in various postures;
[0010] Manually annotate the plurality of historical images to obtain the historical area annotation result.
[0011] In some embodiments, obtaining training data further comprises:
[0012] Performing correction processing on the plurality of historical images to obtain a plurality of corrected historical images;
[0013] Manually labeling the plurality of historical images to obtain the historical area labeling result includes:
[0014] A predetermined segmentation algorithm is used to perform region segmentation on each of the objects in the plurality of corrected historical images, and the segmentation results are manually adjusted to obtain the historical region labeling results.
[0015] In some embodiments, the step of training a preselected large model using the training data to obtain a target large model comprises:
[0016] Inputting the training data into the pre-selected large model for processing to obtain a prediction region detection result corresponding to the corrected historical image;
[0017] Calculating the difference between the predicted area detection result and the historical area annotation result, wherein the difference is characterized by a predetermined loss function;
[0018] Based on the difference, optimizing the model parameters of the preselected large model using a predetermined optimization algorithm;
[0019] Return to the step of inputting the training data into the preselected large model for processing to obtain the predicted area detection result corresponding to the corrected historical image until the predetermined loss function converges. When the predetermined loss function converges, the model parameters of the preselected large model are used as the model parameters of the target large model.
[0020] In some embodiments, the object region detection method further includes, before processing the image to be detected using the target large model to obtain the target region detection result of the object in the image to be detected:
[0021] Performing correction processing on the image to be detected to obtain a corrected image to be detected;
[0022] The step of processing the image to be detected by using the target large model to obtain a target area detection result of the object in the image to be detected includes:
[0023] The target large model is used to process the corrected image to be detected to obtain a target area detection result of the object in the image to be detected.
[0024] In some embodiments, the object includes a stone, and the object region detection method further includes:
[0025] Determine the number of pixels occupied by the stone corresponding to the target area detection result in the image to be detected according to the target area detection result;
[0026] The associated parameters of the stone are calculated according to the number of pixels and the pixel accuracy of the image acquisition device that acquires the image to be detected, and the associated parameters include at least one of area and centroid.
[0027] In some embodiments, the historical image and the image to be detected are both two-dimensional images; or
[0028] The historical image and the image to be detected both include a two-dimensional image and a three-dimensional image, wherein the three-dimensional image is acquired by a depth image acquisition device.
[0029] The object region detection device provided in the embodiment of the present application includes:
[0030] A first acquisition module, the first acquisition module is used to acquire training data, the training data including historical images containing the object and historical region annotation results of the object corresponding to the historical images;
[0031] A training module, the training module is used to train a pre-selected large model using the training data to obtain a target large model;
[0032] A second acquisition module, the second acquisition module is used to acquire an image to be detected containing the object;
[0033] A processing module is used to process the image to be detected using the target large model to obtain a target area detection result of the object in the image to be detected.
[0034] The object region detection system provided in the embodiments of the present application includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the object region detection method described in any of the aforementioned embodiments is implemented.
[0035] The computer-readable storage medium provided in the embodiments of the present application stores a computer program. When the program is executed by a processor, the object region detection method described in any of the aforementioned embodiments is implemented.
[0036] The object region detection method, object region detection device, object region detection system and computer-readable storage medium of the embodiment of the present application use training data including historical images containing objects and historical region annotation results to train a large model to obtain a target large model that matches the region detection of the object, and use the target large model to perform object region detection on the real-time image to be detected to obtain the object region detection result. Compared with the traditional deep learning network, the large model has a more sophisticated network structure, a larger training data set, and a higher training task difficulty. Therefore, the large model has stronger recognition ability and stability, and the accuracy and stability of the object region detection results are higher. In addition, compared with the traditional object region detection algorithm, the large model can use fewer parameters to obtain more generalized and more stable region detection results, which to a certain extent makes up for the deficiency of the traditional object region detection method that cannot distinguish overlapping objects, and better realizes the object detection function of complex scenes.
[0037] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0039] Figure 1 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0040] Figure 2 is a schematic diagram of the working process of the object region detection method in certain embodiments of the present application;
[0041] Figure 3 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0042] Figure 4 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0043] Figure 5 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0044] Figure 6 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0045] Figure 7 is a flowchart of an object region detection method according to certain embodiments of the present application;
[0046] Figure 8is a module schematic diagram of an object region detection device in certain embodiments of the present application;
[0047] Fig. 9 is a module schematic diagram of an object region detection device in certain embodiments of the present application;
[0048] Fig.10 is a module schematic diagram of an object region detection device in certain embodiments of the present application;
[0049] Fig.11 is a module schematic diagram of an object region detection system in certain embodiments of the present application;
[0050] Fig.12 It is a schematic diagram of the connection status between a computer-readable storage medium and a processor in certain embodiments of the present application. DETAILED DESCRIPTION
[0051] The following is a further description of the embodiments of the present application in conjunction with the accompanying drawings. The same or similar reference numerals in the accompanying drawings represent the same or similar elements or elements with the same or similar functions from beginning to end. In addition, the embodiments of the present application described below in conjunction with the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the present application.
[0052] See also Figure 1 and Figure 2 , the embodiment of the present application provides an object region detection method. The object region detection method includes:
[0053] 01: Obtain training data, which includes historical images containing objects and historical region annotation results of objects corresponding to the historical images;
[0054] 02: Use the training data to train the pre-selected large model to obtain the target large model;
[0055] 03: Get the image to be detected containing the object;
[0056] 04: Use the target large model to process the image to be detected to obtain the target area detection result of the object in the image to be detected.
[0057] The object region detection method of the embodiment of the present application can be used to detect objects such as stones, mechanical parts, etc. The present application is described using stones as an example, but this should not be construed as a limitation to the present application.
[0058] A large model refers to a "large parameter" model (i.e., a large-scale deep learning model) trained using large-scale data and powerful computing power. The number of parameters in a large model can reach hundreds of millions, billions, or even more. These models can process complex visual information and adapt to changing scenes. The large model selected in the implementation mode of the present application can be a visual large model, and a trained large model can be selected from an open source data set as a backbone network.
[0059] In step 01, the historical images in the training data can be collected by an image acquisition device. Alternatively, the historical images can be directly obtained from an established database containing multiple stone images. The number of historical images can include multiple images, and the multiple historical images can be images taken for stones of various shapes, and can be images taken for stones in various postures. By collecting stone images containing various shapes and postures, richer model inputs can be provided in subsequent model training, and the trained large model can be adapted to stone area detection in more situations, and the large model has stronger scene adaptability and higher detection accuracy.
[0060] In addition, as an example, the historical image can be a two-dimensional image, such as a two-dimensional color image, a two-dimensional grayscale image, etc., which is not limited in this application. As another example, the historical image can include both a two-dimensional image and a three-dimensional image. The two-dimensional image can be acquired by an ordinary image acquisition device that can obtain color confidence or grayscale information, and the three-dimensional image can be acquired by a depth image acquisition device. For example, the depth image acquisition device can be a time-of-flight camera, a structured light camera, etc., which is not limited in this application. Providing two-dimensional images and three-dimensional images as training data at the same time, the three-dimensional image can provide depth information, which is beneficial to improving the training effect of the target large model, and is also beneficial to improving the accuracy of object area detection.
[0061] The historical region labeling result of the object corresponding to the historical image can be obtained by manually labeling the object in each historical image in advance, or by first performing region segmentation using a predetermined segmentation algorithm and then manually adjusting it.
[0062] As an example, see Figure 3 , obtaining training data may include:
[0063] 011: Get multiple historical images;
[0064] 012: Manually annotate multiple historical images to obtain historical area annotation results.
[0065] In this example, multiple historical images are manually annotated to obtain historical area annotation results. Since objects in historical images usually have complex shapes, textures, and backgrounds, it may be difficult for automatic segmentation algorithms to accurately identify all objects. However, manual annotation can be used to carefully and accurately annotate each object in the historical image based on professional knowledge and experience, thereby ensuring the accuracy of the annotation results. Furthermore, when the target large model is obtained by training with training data, the prediction accuracy and reliability of the target large model can be improved, and the target large model can be adapted to complex tasks.
[0066] In step 02, the historical images and historical region annotation results in the training data can be used as inputs to the pre-selected large model. During the training process, the large model will output the predicted results corresponding to the historical images, and the historical region annotation results will be used as the actual results. The difference between the predicted results and the actual results is then compared, and the parameters of the large model are continuously adjusted based on the difference. In this way, after multiple iterations of training, the difference between the predicted results and the actual results becomes smaller and smaller, thereby training a target large model with a higher accuracy of object region detection results.
[0067] In step 03, as an example, obtaining the image to be detected may be taking a single image of the stone pile to be detected as the image to be detected. As another example, obtaining the image to be detected may also be taking multiple images of the stone pile to be detected as the images to be detected, wherein the multiple images are images collected from different angles. The present application is not limited to this. It is understandable that by taking multiple images collected from different angles as images to be detected and processing them using a trained large target model, multiple target area detection results can be obtained. Combining multiple target area detection results is conducive to obtaining more accurate area detection results.
[0068] It should be noted that the image to be detected may be a two-dimensional image, or may include both a two-dimensional image and a three-dimensional image. Specifically, when the historical image is a two-dimensional image, the image to be detected is a two-dimensional image. When the historical image includes both a two-dimensional image and a three-dimensional image, the image to be detected also includes both a two-dimensional image and a three-dimensional image.
[0069] In step 04, the image to be detected is input into the target large model, and processed by the target large model to obtain a detection result, which is the target area detection result.
[0070] The object region detection method of the embodiment of the present application uses training data containing historical images of objects and historical region annotation results to train a large model to obtain a target large model that matches the region detection of the object, and uses the target large model to perform object region detection on the real-time image to be detected to obtain the object region detection result. Compared with the traditional deep learning network, the large model has a more sophisticated network structure, a larger training data set, and a higher training task difficulty. Therefore, the large model has stronger recognition ability and stability, and the accuracy and stability of the object region detection results are higher. In addition, compared with the traditional object region detection algorithm, the large model can use fewer parameters to obtain more generalized and more stable region detection results, which to a certain extent makes up for the deficiency of the traditional object region detection method that cannot distinguish overlapping objects, and better realizes the object detection function of complex scenes.
[0071] See also Figure 4 In some embodiments, obtaining training data further includes:
[0072] 013: Perform correction processing on a plurality of historical images to obtain a plurality of corrected historical images;
[0073] Manually annotate multiple historical images to obtain historical area annotation results, including:
[0074] 0121: Use a predetermined segmentation algorithm to perform regional segmentation on each object in the multiple corrected historical images, and manually adjust the segmentation results to obtain historical area labeling results.
[0075] It is understandable that, since the lens and image sensor of the image acquisition device are not completely parallel, the image acquired by the image acquisition device may be tangentially distorted. Moreover, it is difficult to ensure uniform thickness of optical lenses during the manufacturing process, wherein the farther away from the center of the lens, the greater the bending of the light, which will cause radial distortion of the image. In addition, when the image acquisition device acquires the image of the stone, it is at a certain distance from the stone. In order to accurately calculate the area, center of mass and other parameters of each stone in the subsequent calculation of the associated parameters of the stone, it is necessary to determine the correspondence between a single pixel on the image and the actual physical distance, that is, to determine the pixel accuracy of the image. Therefore, in this embodiment, it is necessary to perform distortion correction and calibration processing on multiple historical images to obtain corrected historical images. Corrected historical images eliminate the influence of distortion and clarify their pixel accuracy, which is conducive to subsequent processing.
[0076] In this example, the predetermined segmentation algorithm may be an instance segmentation task. The instance segmentation task is an advanced task in computer vision that combines the characteristics of target detection and semantic segmentation. The goal is to accurately segment the pixel area of each target object in the image and distinguish different object instances. By using the instance segmentation task to perform regional segmentation on each stone in multiple corrected historical images, a historical region labeling result for each stone in each corrected historical image can be obtained. In other examples, the predetermined segmentation algorithm may also be other algorithms, which is not limited in this application. After using the predetermined segmentation algorithm to perform regional segmentation on the historical corrected image, the segmentation result may also be manually adjusted to obtain a historical region labeling result with higher accuracy.
[0077] See also Figure 5 In some embodiments, training a preselected large model using training data to obtain a target large model includes:
[0078] 021: Input the training data into the pre-selected large model for processing to obtain the predicted area detection result corresponding to the corrected historical image;
[0079] 022: Calculate the difference between the predicted area detection result and the historical area annotation result, and the difference is characterized by a predetermined loss function;
[0080] 023: Determine whether the predetermined loss function converges;
[0081] 024: If the predetermined loss function converges, the model parameters of the large model pre-selected when the predetermined loss function converges are used as the model parameters of the target large model;
[0082] 025: If the predetermined loss function does not converge, based on the difference, a predetermined optimization algorithm is used to optimize the model parameters of the pre-selected large model, and the step of returning to input the training data into the pre-selected large model for processing to obtain a prediction area detection result corresponding to the corrected historical image.
[0083] In this embodiment, after selecting the large model, the training data is input into the large model to obtain the prediction region detection result corresponding to each corrected historical image. Each prediction region detection result is compared with the historical region annotation result corresponding to each corrected historical image to obtain the difference between the two, and the multiple differences obtained are measured by using a predetermined loss function. Subsequently, it is determined whether the output value of the predetermined loss function is small enough, and when the output value of the predetermined loss function is small enough, the model parameters of the large model of the current training cycle are directly used as the model parameters of the target large model. If the output value of the predetermined loss function is large, based on the difference, the model parameters of the large model of the current training cycle are optimized using a predetermined optimization algorithm. Subsequently, the training data is input into the large model after parameter adjustment to obtain the prediction region detection result corresponding to each corrected historical image again. Subsequently, the value of the predetermined loss function is obtained based on the difference between each prediction region detection result and the corresponding historical region annotation result. It is determined whether the predetermined loss function converges, and when it converges, the model parameters of the large model of the current training cycle are used as the model parameters of the target large model. When the predetermined loss function does not converge, the model parameters of the large model are continuously optimized. This cycle is repeated until the predetermined loss function converges. At this time, the model parameters of the large model when the predetermined loss function converges are used as the model parameters of the target large model to complete the training of the large model.
[0084] The predetermined loss function may be a cross entropy loss function, etc., which is not limited in this application. The predetermined optimization algorithm may be a stochastic gradient descent algorithm, etc., which is not limited in this application.
[0085] See also Figure 6 In some embodiments, before the object region detection method uses the target large model to process the image to be detected to obtain the target region detection result of the object in the image to be detected, it also includes:
[0086] 05: Correct the image to be detected to obtain a corrected image to be detected;
[0087] The target large model is used to process the image to be detected to obtain the target area detection result of the object in the image to be detected, including:
[0088] 041: Use the target large model to process the corrected image to be detected to obtain the target area detection result of the object in the image to be detected.
[0089] As mentioned above, the image captured by the image acquisition device may be distorted. Therefore, in this embodiment, it is necessary to perform correction processing on the image to be detected to eliminate the influence of distortion on the image to be detected, so as to obtain the corrected image to be detected. In addition, it is also necessary to synchronously calibrate the pixel accuracy of the image acquisition device that captures the image to be detected. Subsequently, the corrected image to be detected is input into the target large model to obtain the target area detection result. In this way, a more accurate target area detection result can be obtained.
[0090] See also Figure 7 In some embodiments, the object includes a stone, and the object region detection method further includes:
[0091] 06: Determine the number of pixels occupied by the stone corresponding to the target area detection result in the image to be detected according to the target area detection result;
[0092] 07: Calculate the associated parameters of the stone according to the number of pixels and the pixel accuracy of the image acquisition device that acquires the image to be detected, and the associated parameters include at least one of the area and the centroid.
[0093] In this embodiment, based on the target area detection result output by the target large model, the area information of each stone and the number of pixels occupied by each stone on the image can be easily identified on the stone image. Subsequently, combined with the pixel accuracy of the image acquisition device that acquires the image to be detected, the associated parameters of each stone can be calculated. As an example, the associated parameters include the area and centroid of each stone at the same time. Optionally, step 06 and step 07 can be implemented with the help of Blob (Binary Large Object) analysis algorithm, that is, Blob analysis can be used to perform area identification on the target area detection result output by the target large model and calculate the associated parameters of the stone. In this way, after obtaining the area and centroid of each stone, the quality of each stone can be accurately determined based on this information, as well as the quality of the stone pile. By processing the stone image in combination with the large model, each stone can be accurately measured, which is conducive to improving the accuracy of the quality determination of the stone and improving the efficiency of the quality determination of the stone.
[0094] See also Figure 8The embodiment of the present application also provides an object region detection device 10. The object region detection device 10 includes a first acquisition module 11, a training module 12, a second acquisition module 13 and a processing module 14. The first acquisition module 11 is used to acquire training data, and the training data includes historical images containing objects and historical region annotation results of objects corresponding to the historical images. The training module 12 is used to train a pre-selected large model using the training data to obtain a target large model. The second acquisition module 13 is used to acquire an image to be detected containing an object. The processing module 14 is used to process the image to be detected using the target large model to obtain a target region detection result of the object in the image to be detected.
[0095] In some embodiments, the first acquisition module 11 is further used to acquire multiple historical images and manually annotate the multiple historical images to obtain historical area annotation results. The multiple historical images are images acquired by capturing objects of various shapes and in various postures.
[0096] In some embodiments, the first acquisition module 11 is further used to perform correction processing on multiple historical images to obtain multiple corrected historical images, use a predetermined segmentation algorithm to perform region segmentation on each object in the multiple corrected historical images, and manually adjust the segmentation results to obtain historical area labeling results.
[0097] In some embodiments, the training module 12 is further used to: input the training data into a pre-selected large model for processing to obtain a predicted region detection result corresponding to the corrected historical image; calculate the difference between the predicted region detection result and the historical region annotation result, and the difference is characterized by a predetermined loss function; based on the difference, optimize the model parameters of the pre-selected large model using a predetermined optimization algorithm; return to execute the step of inputting the training data into the pre-selected large model for processing to obtain a predicted region detection result corresponding to the corrected historical image, until the predetermined loss function converges. When the loss function converges, the model parameters of the pre-selected large model are used as the model parameters of the target large model.
[0098] See also Fig. 9 In some embodiments, the object region detection device 10 further includes a correction module 15. The correction module 15 is used to perform correction processing on the image to be detected to obtain a corrected image to be detected. The processing module 14 is further used to process the corrected image to be detected using the target large model to obtain a target region detection result of the object in the image to be detected.
[0099] See also Fig.10In some embodiments, the object region detection device 10 further includes a determination module 16 and a calculation module 17. The determination module 16 is used to determine the number of pixels occupied by the stone corresponding to the target region detection result in the image to be detected according to the target region detection result. The calculation module 17 is used to calculate the associated parameters of the stone according to the number of pixels and the pixel accuracy of the image acquisition device that acquires the image to be detected. The associated parameters include at least one of the area and the centroid.
[0100] It should be pointed out that the explanation of the object region detection method in the aforementioned embodiment is also applicable to the object region detection device 10 in the embodiment of the present application, and will not be elaborated here.
[0101] See also Fig.11 The embodiment of the present application further provides an object region detection system 20. The object region detection system 20 includes one or more processors 21 and a memory 22. The memory 22 stores a computer program, and when the computer program is executed by the processor 21, the object region detection method of any of the above embodiments is implemented.
[0102] For example, when the computer program is executed by the processor 21, the following object region detection method is implemented:
[0103] 01: Obtain training data, which includes historical images containing objects and historical region annotation results of objects corresponding to the historical images;
[0104] 02: Use the training data to train the pre-selected large model to obtain the target large model;
[0105] 03: Get the image to be detected containing the object;
[0106] 04: Use the target large model to process the image to be detected to obtain the target area detection result of the object in the image to be detected.
[0107] For another example, when the computer program is executed by the processor 21, the following object region detection method is implemented:
[0108] 011: Get multiple historical images;
[0109] 012: Manually annotate multiple historical images to obtain historical area annotation results.
[0110] It should be pointed out that the explanations and descriptions of the object region detection method and the object region detection device 10 in the aforementioned embodiments are also applicable to the object region detection system 20 in the embodiments of the present application, and will not be elaborated here.
[0111] See also Fig.12The present application also provides a computer-readable storage medium 30 on which a computer program is stored. When the computer program is executed by the processor 40, the object region detection method of any of the above embodiments is implemented.
[0112] For example, when the computer program is executed by the processor 40, the following object region detection method is implemented:
[0113] 01: Obtain training data, which includes historical images containing objects and historical region annotation results of objects corresponding to the historical images;
[0114] 02: Use the training data to train the pre-selected large model to obtain the target large model;
[0115] 03: Get the image to be detected containing the object;
[0116] 04: Use the target large model to process the image to be detected to obtain the target area detection result of the object in the image to be detected.
[0117] For another example, when the computer program is executed by the processor 40, the following object region detection method is implemented:
[0118] 011: Get multiple historical images;
[0119] 012: Manually annotate multiple historical images to obtain historical area annotation results.
[0120] It should be pointed out that the explanations and descriptions of the object region detection method and the object region detection device in the aforementioned embodiments are also applicable to the computer-readable storage medium in the embodiments of the present application and will not be elaborated here.
[0121] In summary, the object region detection method, object region detection device 10, object region detection system 20 and computer-readable storage medium 30 of the embodiment of the present application use training data including historical images containing objects and historical region annotation results to train a large model to obtain a target large model that matches the region detection of the object, and use the target large model to perform object region detection on the real-time image to be detected to obtain the object region detection result. Compared with the traditional deep learning network, the large model has a more sophisticated network structure, a larger training data set, and a higher training task difficulty. Therefore, the large model has stronger recognition ability and stability, and the accuracy and stability of the object region detection results are higher. Moreover, compared with the traditional object region detection algorithm, the large model can use fewer parameters to obtain more generalized and more stable region detection results, which to a certain extent makes up for the deficiency of the traditional object region detection method that cannot distinguish overlapping objects, and better realizes the object detection function of complex scenes.
[0122] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0123] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a computer-readable storage medium can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0125] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0126] A person of ordinary skill in the art will appreciate that all or part of the steps carried by the method for implementing the above-mentioned embodiment can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment. In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.
[0127] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application. The scope of the present application is defined by the claims and their equivalents.
Claims
1. A method for detecting an object region, characterized in that: The object area detection method comprises: Acquire training data, the training data including historical images containing the object and historical region annotation results of the object corresponding to the historical images; Using the training data to train a preselected large model to obtain a target large model; Acquire an image to be detected containing the object; The target large model is used to process the image to be detected to obtain a target area detection result of the object in the image to be detected.
2. The object region detection method according to claim 1, characterized in that: The obtaining of training data comprises: Acquire a plurality of the historical images, where the plurality of the historical images are images acquired by capturing images of the object in various shapes and in various postures; Manually annotate the plurality of historical images to obtain the historical area annotation result.
3. The object region detection method according to claim 2, characterized in that: The obtaining of training data further comprises: Performing correction processing on the plurality of historical images to obtain a plurality of corrected historical images; Manually labeling the plurality of historical images to obtain the historical area labeling result includes: A predetermined segmentation algorithm is used to perform region segmentation on each of the objects in the plurality of corrected historical images, and the segmentation results are manually adjusted to obtain the historical region labeling results.
4. The object region detection method according to claim 3, characterized in that: The method of training the pre-selected large model using the training data to obtain a target large model includes: Inputting the training data into the pre-selected large model for processing to obtain a prediction region detection result corresponding to the corrected historical image; Calculating the difference between the predicted area detection result and the historical area annotation result, wherein the difference is characterized by a predetermined loss function; Based on the difference, optimizing the model parameters of the preselected large model using a predetermined optimization algorithm; Return to the step of inputting the training data into the preselected large model for processing to obtain the predicted area detection result corresponding to the corrected historical image until the predetermined loss function converges. When the predetermined loss function converges, the model parameters of the preselected large model are used as the model parameters of the target large model.
5. The object region detection method according to claim 1, characterized in that: The object region detection method further includes, before using the target large model to process the image to be detected to obtain the target region detection result of the object in the image to be detected: Performing correction processing on the image to be detected to obtain a corrected image to be detected; The step of processing the image to be detected by using the target large model to obtain a target area detection result of the object in the image to be detected includes: The target large model is used to process the corrected image to be detected to obtain a target area detection result of the object in the image to be detected.
6. The object region detection method according to claim 1, characterized in that: The object includes a stone, and the object area detection method further includes: Determine the number of pixels occupied by the stone corresponding to the target area detection result in the image to be detected according to the target area detection result; The associated parameters of the stone are calculated according to the number of pixels and the pixel accuracy of the image acquisition device that acquires the image to be detected, and the associated parameters include at least one of area and centroid.
7. The object region detection method according to claim 1, characterized in that: The historical image and the image to be detected are both two-dimensional images; or The historical image and the image to be detected both include a two-dimensional image and a three-dimensional image, wherein the three-dimensional image is acquired by a depth image acquisition device.
8. An object region detection device, characterized in that: The object area detection device comprises: A first acquisition module, the first acquisition module is used to acquire training data, the training data including historical images containing the object and historical region annotation results of the object corresponding to the historical images; A training module, the training module is used to train a pre-selected large model using the training data to obtain a target large model; A second acquisition module, the second acquisition module is used to acquire an image to be detected containing the object; A processing module is used to process the image to be detected using the target large model to obtain a target area detection result of the object in the image to be detected.
9. An object region detection system, characterized in that: The object region detection system includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the object region detection method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the object area detection method described in any one of claims 1 to 7 is implemented.