Method and device for detecting master-slave relation between objects in image, equipment and medium

Abstract

The invention discloses a method, a device and equipment for detecting a master-slave relationship between objects in an image and a medium, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring pixel information of a to-be-detected image, and constructing an equal-resolution canvas according to the pixel information; identifying region information of at least one main object and region information of at least one subordinate object in the to-be-detected image, and mapping the region information of the main object and the region information of the subordinate object to a canvas; and according to a mapping result of the main body object and the subordinate object in the canvas, establishing a master-slave relationship between the main body object and the subordinate object. According to the technical scheme, the regional information of the main object and the subordinate object is identified and mapped to the canvas, and the master-slave relationship between the main object and the subordinate object is established based on the mapping result, so that the influence of perspective distortion of the to-be-detected image and distortion of the camera can be remarkably reduced; and the detection efficiency and accuracy of the master-slave relationship between the main object and the slave object are improved.

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and specifically relates to a method, device, equipment and medium for detecting a master-slave relationship between objects in an image. Background Art

[0002] In the field of computer vision, determining the master-slave relationship between objects in an image is of great significance. In particular, when there are multiple objects with exactly the same appearance in an image, accurately identifying the master-slave relationship between these objects and other objects can effectively distinguish these objects, thereby significantly improving the semantic understanding and decision-making capabilities of the computer vision system.

[0003] In the prior art, the main method of determining the master-slave relationship is to detect closed contours, but this method is easily affected by factors such as exposure, contrast, and object occlusion, making it difficult to separate accurate contours. In addition, there is also a method that combines image correction algorithms and target detection algorithms to determine the master-slave relationship, but the camera's own distortion and perspective distortion will cause information bias, which can easily cause the subordinate object to be attributed to the wrong main object, and the accuracy is low.

[0004] Therefore, how to overcome the above defects and accurately determine the master-slave relationship between objects in an image is a problem that people in this field need to solve urgently. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device and medium for detecting a master-slave relationship between objects in an image, the purpose of which is to significantly reduce the influence of perspective distortion of the image to be detected and the distortion of the camera itself, and to improve the detection efficiency and accuracy of the master-slave relationship between the main object and the subordinate object.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting a master-slave relationship between objects in an image, the method comprising:

[0007] Obtaining pixel information of the image to be detected, and constructing a canvas of equal resolution according to the pixel information;

[0008] Identifying region information of at least one main object in the image to be detected, and mapping the region information of the main object into the canvas;

[0009] Identifying region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object into the canvas;

[0010] A master-slave relationship between the main object and the subordinate object is established according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

[0011] In a second aspect, an embodiment of the present application provides a device for detecting a master-slave relationship between objects in an image, the device comprising:

[0012] A canvas construction module, used to obtain pixel information of the image to be detected and construct a canvas of equal resolution according to the pixel information;

[0013] A subject recognition and mapping module, used to recognize the region information of at least one subject object in the image to be detected, and map the region information of the subject object into the canvas;

[0014] A subordinate identification and mapping module, used for identifying the region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object into the canvas;

[0015] The master-slave relationship establishment module is used to establish a master-slave relationship between the main object and the subordinate object according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0019] In an embodiment of the present application, pixel information of an image to be detected is obtained, and a canvas of equal resolution is constructed based on the pixel information; regional information of at least one main object in the image to be detected is identified, and the regional information of the main object is mapped to the canvas; regional information of at least one subordinate object in the image to be detected is identified, and the regional information of the subordinate object is mapped to the canvas; a master-slave relationship between the main object and the subordinate object is established based on the mapping result of the main object and the mapping result of the subordinate object in the canvas. The above-mentioned master-slave relationship detection method between objects in an image, by identifying the regional information of the main object and the subordinate object and mapping them to the canvas, and establishing the master-slave relationship between the main object and the subordinate object based on the mapping result, can significantly reduce the influence of perspective distortion of the image to be detected and the distortion of the camera itself, and improve the detection efficiency and accuracy of the master-slave relationship between the main object and the subordinate object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a method for detecting a master-slave relationship between objects in an image provided in Embodiment 1 of the present application;

[0021] Figure 2 is a flow chart of a method for detecting a master-slave relationship between objects in an image provided in Embodiment 2 of the present application;

[0022] Figure 3 is an example diagram of an image to be detected provided in Embodiment 2 of the present application;

[0023] Figure 4 This is an example diagram of the mapping result of the main object in the canvas provided in the second embodiment of the present application;

[0024] Figure 5 is a flowchart of a method for detecting a master-slave relationship between objects in an image provided in Embodiment 3 of the present application;

[0025] Figure 6 This is an example diagram of the mapping results of the main object and the mapping results of the subordinate objects in the canvas provided in the third embodiment of the present application;

[0026] Figure 7 It is a partial example diagram of the mapping results of the main object and the mapping results of the subordinate objects in the canvas provided in the third embodiment of the present application;

[0027] Figure 8 is a schematic diagram of the structure of a device for detecting a master-slave relationship between objects in an image provided by Embodiment 4 of the present application;

[0028] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0031] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0032] In conjunction with the accompanying drawings, the following detailed description is given of the master-slave relationship detection method, device, equipment and medium between objects in an image provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0033] Embodiment 1

[0034] Figure 1 FIG. 1 is a flow chart of a method for detecting a master-slave relationship between objects in an image provided by Embodiment 1 of the present application. Figure 1 As shown, the specific steps include:

[0035] S101, obtaining pixel information of an image to be detected, and constructing a canvas of equal resolution according to the pixel information;

[0036] First of all, the present application is applicable to scenarios where it is necessary to detect the master-slave relationship between objects in an image, specifically, for example, there are multiple main objects and multiple subordinate objects in the image, and it is necessary to determine the semantic association between each main object and each subordinate object. Based on the above usage scenarios, it can be understood that the execution subject of the present application can be an intelligent terminal device with image recognition and data processing capabilities, such as: mobile phones, tablet computers and desktop computers, etc., which are not too limited here. Specifically, the construction of the canvas, the identification and mapping of the regional information of the main object and the subordinate object, and the establishment of the master-slave relationship between the main object and the subordinate object can be executed by the intelligent terminal device, and the master-slave relationship between the main object and the subordinate object can be established to help realize the corresponding business functions.

[0037] The image to be detected may refer to an image in which a master-slave relationship between a main object and a subordinate object needs to be established. An image is a two-dimensional information carrier composed of many pixels, and a pixel is the smallest unit in an image. The pixel information of the image to be detected is information used to describe each pixel in the image to be detected, and the pixel information of the image to be detected can be directly read and obtained.

[0038] A canvas can be understood as an image that represents an empty content. An equal-resolution canvas can mean that the resolution of the constructed canvas is consistent with the resolution of the image to be detected. The resolution can refer to the number of pixels contained in the image, usually expressed as horizontal pixels × vertical pixels.

[0039] The method of constructing a canvas of equal resolution according to pixel information can be adopted to determine the resolution of the image to be detected according to the pixel information of the image to be detected, and to create a new blank image as a canvas of equal resolution according to the resolution of the image to be detected.

[0040] In addition to constructing a canvas of equal resolution based on pixel information, a single-channel image of the image to be detected can be output as a canvas of equal resolution based on the pixel information of the image to be detected. Specifically, a Mat type object can be created through the OpenCV library to store the image to be detected, and a single-channel image of equal resolution to the image to be detected can be created as a canvas of equal resolution through the create function of the Mat type object.

[0041] S102, identifying region information of at least one main object in the image to be detected, and mapping the region information of the main object to the canvas;

[0042] The main object can be the basis for the existence of the subordinate object. The subordinate object may not exist on the main object, but the subordinate object must exist on a main object. For example, the image to be detected is an image of an electric room acquired by a surveillance camera. In this image of the electric room, the main object can be an electric cabinet panel, and the subordinate object can be a digital instrument device.

[0043] The area information of the main object can be information describing the pixel area occupied by the main object in the image to be detected. Among them, the area information of the main object can include mask information and / or border information of the main object. Specifically, the mask information refers to a two-dimensional matrix of the same size as the image to be detected that can represent the object area, the pixel value of the object position is a specific value (usually 1 or 255), and the pixel value of the remaining position is another specific value (usually 0); the border information refers to a rectangular frame that can frame the object. Accordingly, the Mat type object in the OpenCV library can be used to store the mask information, and the Rect type object in the OpenCV library can be used to store the border information.

[0044] The method for identifying the regional information of at least one main object in the image to be detected can be to input the image to be detected into a pre-trained first instance segmentation model, and determine the regional information of at least one main object in the image to be detected based on the output result of the first instance segmentation model.

[0045] The method of mapping the area information of the main object to the canvas can be to determine the pixel area corresponding to the main object in the canvas according to the area information of the main object, and according to the association between the main object and the pixel value, set the pixel value of each pixel in the pixel area to the pixel value associated with the main object to obtain the mapping result of the main object.

[0046] S103, identifying region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object to the canvas;

[0047] The region information of the subordinate object may be information describing the pixel region occupied by the subordinate object in the image to be detected, wherein the region information of the subordinate object may include mask information and / or frame information of the subordinate object.

[0048] In this solution, it is preferred that the region information of the main object adopts mask information and the region information of the subordinate object adopts frame information, or the region information of the main object adopts frame information and the region information of the subordinate object adopts mask information.

[0049] The method for identifying the regional information of at least one subordinate object in the image to be detected can be that the image to be detected is input into a pre-trained second instance segmentation model, and the regional information of at least one subordinate object in the image to be detected is determined according to the output result of the second instance segmentation model.

[0050] The method of mapping the region information of the subordinate object to the canvas may be to determine the pixel region occupied by the subordinate object in the canvas according to the region information of the subordinate object, and obtain the mapping result of the subordinate object.

[0051] In the technical solution, optionally, identifying the area information of at least one main object in the image to be detected includes:

[0052] Inputting the image to be detected into a pre-trained first instance segmentation model, and determining region information of at least one main object in the image to be detected according to an output result of the first instance segmentation model;

[0053] Accordingly, identifying the region information of at least one subordinate object in the image to be detected includes:

[0054] The image to be detected is input into a pre-trained second instance segmentation model, and region information of at least one subordinate object in the image to be detected is determined according to an output result of the second instance segmentation model.

[0055] The first instance segmentation model may be an instance segmentation model for obtaining instances of each subject object in the image to be detected. It is understandable that the first instance segmentation model may be pre-trained with annotated subject object images. The instance segmentation model is a deep learning model in the field of computer vision, which is used to accurately segment different objects in an image and distinguish different instances.

[0056] The image to be detected is input into a pre-trained first instance segmentation model, and the region information of at least one main object in the image to be detected is determined according to the output result of the first instance segmentation model. The image to be detected can be input into a pre-trained first instance segmentation model, and the first instance segmentation model outputs each main object instance in the image to be detected, and the region information of each main object instance is obtained as the region information of each main object.

[0057] The second instance segmentation model may be an instance segmentation model for obtaining instances of each subordinate object in the image to be detected. It is understandable that the second instance segmentation model may be pre-trained with labeled subordinate object images.

[0058] The image to be detected is input into a pre-trained second instance segmentation model, and the region information of at least one subordinate object in the image to be detected is determined according to the output result of the second instance segmentation model. The image to be detected can be input into a pre-trained second instance segmentation model, and the second instance segmentation model outputs each subordinate object instance in the image to be detected, and the region information of each subordinate object instance is obtained as the region information of each subordinate object.

[0059] In the present technical solution, optionally, the first instance segmentation model and the second instance segmentation model respectively adopt Yolov5 instance segmentation model.

[0060] The Yolov5 instance segmentation model is a deep learning model that can perform instance segmentation tasks and is based on the target detection model Yolov5. The Yolov5 instance segmentation model can use CSPDarknet53 as the backbone network and PANet (path aggregation network) as the neck network structure. PANet further enhances the feature fusion effect and transmits the low-level positioning information to the high-level through a bottom-up path, thus improving the segmentation accuracy of the model for small targets.

[0061] Correspondingly, the method of pre-training the first instance segmentation model can use the labeled main object images as training samples, set the training sample batch size to 32, set the initial learning rate to 0.001, and use the Adam optimizer to adaptively adjust the learning rate of each parameter by calculating the first-order moment estimate and the second-order moment estimate of the gradient to balance the convergence speed and stability. Bounding box regression, classification loss, and mask loss dedicated to segmentation are selected as loss functions to improve the segmentation performance of the first instance segmentation model.

[0062] Correspondingly, the method of pre-training the second instance segmentation model can be to select labeled subordinate object images as training samples, set the training sample batch size to 32, set the initial learning rate to 0.001, and select the Adam optimizer to adaptively adjust the learning rate of each parameter by calculating the first-order moment estimate and the second-order moment estimate of the gradient to balance the convergence speed and stability. Bounding box regression, classification loss, and mask loss dedicated to segmentation are selected as loss functions to improve the segmentation performance of the second instance segmentation model.

[0063] The advantage of this arrangement of the present invention is that, by using the Yolov5 instance segmentation model respectively through the first instance segmentation model and the second instance segmentation model, the image to be detected can be processed and analyzed quickly, which is particularly suitable for scenarios such as video surveillance and autonomous driving.

[0064] The advantage of this arrangement of the present scheme is that by using the first instance segmentation model to identify the regional information of each main object in the image to be detected, and by using the second instance segmentation model to identify the regional information of each subordinate object in the image to be detected, independent analysis and processing of the main object and the subordinate object can be achieved, thereby more accurately determining the regional information of the main object and the subordinate object in the image to be detected.

[0065] S104: establishing a master-slave relationship between the main object and the subordinate objects according to the mapping results of the main object and the subordinate objects in the canvas.

[0066] The master-slave relationship between the main object and the subordinate object can be used to describe the subordinate attribute of a subordinate object to a main object. It can be understood that, according to the master-slave relationship between the main object and the subordinate object, the subordinate objects included in a main object can be determined, and the main object to which a subordinate object is subordinate can also be determined. For example, the main object is an electric cabinet panel, and the subordinate object is a digital instrument device. According to the master-slave relationship between the electric cabinet panel and the digital instrument device, the digital instrument devices installed on an electric cabinet panel can be determined, and the electric cabinet panel where a digital instrument device is located can also be determined.

[0067] The master-slave relationship between the main object and the subordinate object can be established by determining the target main object according to the pixel value of each pixel in the mapping result of the subordinate object, and determining that a master-slave relationship exists between the target main object and the subordinate object.

[0068] In an embodiment of the present application, the regional information of at least one main object in the image to be detected is identified, and the regional information of the main object is mapped to the canvas; the regional information of at least one subordinate object in the image to be detected is identified, and the regional information of the subordinate object is mapped to the canvas; and the master-slave relationship between the main object and the subordinate object is established based on the mapping result of the main object and the mapping result of the subordinate object in the canvas. The master-slave relationship detection method between objects in the above-mentioned image can significantly reduce the influence of perspective distortion of the image to be detected and the distortion of the camera itself, and improve the detection efficiency and accuracy of the master-slave relationship between the main object and the subordinate object by identifying the regional information of the main object and the subordinate object and mapping them to the canvas, and establishing the master-slave relationship between the main object and the subordinate object based on the mapping result.

[0069] Embodiment 2

[0070] Figure 2It is a flow chart of the master-slave relationship detection method between objects in an image provided by the second embodiment of the present application. This scheme makes a better improvement on the above embodiment, and the specific improvement is: after obtaining the pixel information of the image to be detected and constructing a canvas of equal resolution according to the pixel information, the method also includes: setting the pixel value of each pixel in the canvas to a preset pixel value; correspondingly, mapping the area information of the main object to the canvas, including: assigning different pixel values ​​to each main object respectively, and obtaining the association relationship between the main object and the pixel value; adjusting the pixel value of each pixel in the canvas according to the area information of the main object and the association relationship between the main object and the pixel value.

[0071] like Figure 2 As shown, the specific steps include:

[0072] S201, obtaining pixel information of an image to be detected, and constructing a canvas of equal resolution according to the pixel information;

[0073] S202, setting the pixel value of each pixel in the canvas to a preset pixel value;

[0074] The preset pixel value may be a preset pixel value indicating that an object does not exist, for example, the preset pixel value is 0. By changing the data indicating the pixel value of each pixel in the canvas to the preset pixel value, the pixel value of each pixel in the canvas may be set to the preset pixel value. Specifically, the pixel value of each pixel in the canvas may be set to the preset pixel value through the setTo function of the Mat type object storing the canvas.

[0075] S203, identifying region information of at least one main object in the image to be detected;

[0076] S204, assigning different pixel values ​​to each main object respectively, and obtaining the association relationship between the main object and the pixel value;

[0077] The association relationship between the main object and the pixel value may refer to a unique correspondence between a main object and a pixel value.

[0078] According to the preset pixel value allocation rule, different pixel values ​​can be allocated to each main object. It can be understood that the pixel value allocated to each main object does not include the preset pixel value. For example, the pixel value corresponding to the first main object is 4, the pixel value corresponding to the second main object is 8, the pixel value corresponding to the third main object is 12, and the subsequent main objects are deduced by analogy.

[0079] By associating and storing the identifier of the subject object and its corresponding pixel value, the association relationship between the subject object and the pixel value can be obtained.

[0080] S205, adjusting the pixel value of each pixel in the canvas according to the region information of the main object and the association relationship between the main object and the pixel value;

[0081] The method of adjusting the pixel value of each pixel in the canvas according to the area information of the main object and the association relationship between the main object and the pixel value can be adopted. According to the area information of the main object, the pixel area corresponding to the main object is determined in the canvas, and according to the association relationship between the main object and the pixel value, the pixel value of each pixel in the pixel area is set to the pixel value associated with the main object.

[0082] In the technical solution, optionally, adjusting the pixel value of each pixel in the canvas according to the area information of the main object and the association relationship between the main object and the pixel value includes:

[0083] Determining a pixel area corresponding to the main object in the canvas according to the area information of the main object;

[0084] According to the association relationship between the main object and the pixel value, the pixel value of each pixel in the pixel area is set to the pixel value associated with the main object.

[0085] The pixel region may refer to a region composed of pixels. In the canvas, the pixel region corresponding to the main object may be determined by traversing the mask information if the region information of the main object is mask information to obtain a pixel coordinate set as the pixel region corresponding to the main object, and by determining the frame length, frame width and frame origin according to the frame information if the region information of the main object is frame information, and determining the pixel region corresponding to the main object according to the frame length, frame width and frame origin.

[0086] By changing the data representing the pixel value of each pixel in the pixel area corresponding to the main object to the pixel value associated with the main object, the pixel value of each pixel in the pixel area can be set to the pixel value associated with the main object. Specifically, the pixel value of each pixel in the pixel area can be set to the pixel value associated with the main object through the setTo function of the Mat type object storing the canvas, such as canvas(mainObjBox).setTo(cv::Scalar(pixelValue), mainObjMask), where canvas is the Mat type object representing the canvas, pixelValue is the pixel value associated with the main object, mainObjBox is the border information of the main object, and mainObjMask is the mask information of the main object.

[0087] Figure 3 is an example of an image to be detected provided in the second embodiment of the present application. Figure 3 As shown, the image to be detected is an image of a power room acquired by a surveillance camera. In this image of the power room, the main object is the electric cabinet panel, and the subordinate object is the digital instrument equipment.

[0088] Figure 4 This is an example diagram of the mapping results of the main object in the canvas provided in Example 2 of the present application. Figure 3 The mapping results of each electrical cabinet panel are as follows Figure 4 As shown in the figure, a total of 5 cabinet panels are identified, and different pixel values ​​are assigned to each cabinet panel, resulting in the 5 cabinet panels being Figure 4 In different colors, Figure 4 The black area in is the area where the pixel value is the preset pixel value.

[0089] The advantage of this setting of the present scheme is that by determining the pixel area corresponding to the main object in the canvas and setting the pixel value of each pixel in the pixel area to the pixel value associated with the main object according to the association between the main object and the pixel value, the specific range and position of the main object can be clearly and accurately identified in the canvas, laying the foundation for the subsequent determination of the target main object of the subordinate object.

[0090] S206, identifying region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object to the canvas;

[0091] S207: Establish a master-slave relationship between the main object and the subordinate object according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

[0092] The advantage of this setting of the present scheme is that by setting the pixel value of each pixel in the canvas to a preset pixel value and assigning different pixel values ​​to each subject object, the pixel areas of different subject objects in the canvas can be clearly distinguished, so that each subject object can be identified and located more accurately and quickly during subsequent processing and analysis.

[0093] Embodiment 3

[0094] Figure 5 It is a flow chart of the master-slave relationship detection method between objects in an image provided by the third embodiment of the present application. This solution makes a better improvement on the above embodiment, and the specific improvement is: according to the mapping result of the main object and the mapping result of the subordinate object in the canvas, a master-slave relationship between the main object and the subordinate object is established, including: according to the pixel value of each pixel in the mapping result of the subordinate object, the target main object is determined; and it is determined that there is a master-slave relationship between the target main object and the subordinate object.

[0095] like Figure 5As shown, the specific steps include:

[0096] S501, obtaining pixel information of an image to be detected, and constructing a canvas of equal resolution according to the pixel information;

[0097] S502, setting the pixel value of each pixel in the canvas to a preset pixel value;

[0098] S503, identifying region information of at least one main object in the image to be detected;

[0099] S504, assigning different pixel values ​​to each main object respectively, and obtaining the association relationship between the main object and the pixel value;

[0100] S505, adjusting the pixel value of each pixel in the canvas according to the region information of the main object and the association relationship between the main object and the pixel value;

[0101] S506, identifying region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object to the canvas;

[0102] S507, determining the target main object according to the pixel value of each pixel in the mapping result of the subordinate object;

[0103] The target main object may refer to the main object to which the subordinate object belongs. According to the pixel value of each pixel in the mapping result of the subordinate object, the method of determining the target main object may be to count the number of pixels of each pixel value in the mapping result of the subordinate object, determine the pixel value with the largest number of pixels as the target pixel value, and determine the target main object according to the target pixel value and the association relationship between the main object and the pixel value.

[0104] In the technical solution, optionally, determining the target main object according to the pixel value of each pixel in the mapping result of the subordinate object includes:

[0105] Counting the number of pixels of each pixel value in the mapping result of the subordinate object;

[0106] Determine the pixel value with the largest number of pixels as the target pixel value;

[0107] The target subject object is determined according to the target pixel value and the association relationship between the subject object and the pixel value.

[0108] The method of counting the number of pixels of each pixel value in the mapping result of the subordinate object may be to sequentially read the pixel value of each pixel in the mapping result of the subordinate object and record the number of pixels with the same pixel value during the reading process.

[0109] Figure 6This is an example diagram of the mapping results of the main object and the subordinate object in the canvas provided in the third embodiment of the present application. Figure 6 As shown, the dotted box is the mapping result of the second digital instrument device.

[0110] Figure 7 This is a partial example diagram of the mapping results of the main object and the subordinate object in the canvas provided in the third embodiment of the present application. Figure 7 As shown, Figure 7 Yes Figure 6 The result of local magnification, the pixel area with a pixel value of 4 is the mapping result of the first electrical cabinet panel, the pixel area with a pixel value of 8 is the mapping result of the second electrical cabinet panel, and the dotted box is the mapping result of the second digital instrument device. There are a total of 18×13=234 pixels in the dotted box, and the pixel values ​​of these 234 pixels are all 8.

[0111] The target pixel value can be regarded as the main pixel value in the mapping result of the subordinate object. The pixel value with the largest number of pixels is determined as the target pixel value by arranging the pixel values ​​in descending order according to the number of pixels, and determining the first pixel value in the arrangement result as the target pixel value. Figure 7 As shown, the pixel value with the largest number of pixels in the dotted box is 8, that is, 8 is determined as the target pixel value.

[0112] According to the target pixel value and the association relationship between the subject object and the pixel value, the target subject object can be determined by using the target pixel value as a query condition to query the stored data of the association relationship between the subject object and the pixel value, and the query result includes the subject target object. For example, using 8 as the query condition, it can be determined that the target subject object is the second electric cabinet panel.

[0113] The advantage of this setting of the present scheme is that by counting the number of pixels of each pixel value in the mapping result of the subordinate object, the pixel value with the largest number of pixels is determined as the target pixel value, and the target main object is determined based on the target pixel value and the association between the main object and the pixel value, the main object that is most closely related to the subordinate object can be found quickly and accurately.

[0114] S508: Determine whether there is a master-slave relationship between the target subject object and the subordinate object.

[0115] The method of determining whether there is a master-slave relationship between the target main object and the subordinate object is to associate and store the target main object with the subordinate object.

[0116] The benefit of this arrangement of the present invention is that by determining the target subject object according to the pixel value of each pixel in the mapping result of the subordinate object and determining that there is a master-slave relationship between the target subject object and the subordinate object, the subject object to which each subordinate object belongs can be accurately determined respectively.

[0117] Embodiment 4

[0118] Figure 8 Schematic diagram of the structure of the master-slave relationship detection device between objects in an image provided by the fourth embodiment of the present application. Figure 8 As shown, the device comprises:

[0119] A canvas construction module 810 is used to obtain pixel information of the image to be detected and construct a canvas of equal resolution according to the pixel information;

[0120] A subject recognition and mapping module 820, configured to recognize region information of at least one subject object in the image to be detected, and map the region information of the subject object to the canvas;

[0121] A subordinate identification and mapping module 830, used to identify the region information of at least one subordinate object in the image to be detected, and map the region information of the subordinate object to the canvas;

[0122] The master-slave relationship establishment module 840 is used to establish a master-slave relationship between the main object and the subordinate object according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

[0123] In the embodiment of the present application, the canvas construction module is used to obtain the pixel information of the image to be detected, and to construct a canvas of equal resolution according to the pixel information; the subject identification mapping module is used to identify the regional information of at least one subject object in the image to be detected, and to map the regional information of the subject object to the canvas; the subordinate identification mapping module is used to identify the regional information of at least one subordinate object in the image to be detected, and to map the regional information of the subordinate object to the canvas; the master-slave relationship establishment module is used to establish the master-slave relationship between the subject object and the subordinate object according to the mapping result of the subject object and the mapping result of the subordinate object in the canvas. The master-slave relationship detection device between objects in the above-mentioned image can significantly reduce the influence of perspective distortion of the image to be detected and the distortion of the camera itself, and improve the detection efficiency and accuracy of the master-slave relationship between the subject object and the subordinate object by identifying the regional information of the subject object and the subordinate object and mapping them to the canvas, and establishing the master-slave relationship between the subject object and the subordinate object based on the mapping result.

[0124] The master-slave relationship detection device between objects in the image in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0125] The master-slave relationship detection device between objects in an image in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0126] The master-slave relationship detection device between objects in an image provided by the embodiment of the present application can implement each process implemented in the above-mentioned embodiments one to three, and will not be described again here to avoid repetition.

[0127] Embodiment 5

[0128] like Fig. 9 As shown, an embodiment of the present application also provides an electronic device 900, including a processor 901, a memory 902, and a program or instruction stored in the memory 902 and executable on the processor 901. When the program or instruction is executed by the processor 901, each process of the embodiment of the master-slave relationship detection device between objects in the above-mentioned image is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0129] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0130] Embodiment 6

[0131] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the embodiment of the master-slave relationship detection device between objects in the above-mentioned image are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0132] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0133] Embodiment 7

[0134] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned master-slave relationship detection device embodiment between objects in the image, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0135] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0136] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0138] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0139] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for detecting a master-slave relationship between objects in an image, characterized in that: The method comprises: Obtaining pixel information of the image to be detected, and constructing a canvas of equal resolution according to the pixel information; Identifying region information of at least one main object in the image to be detected, and mapping the region information of the main object into the canvas; Identifying region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object into the canvas; A master-slave relationship between the main object and the subordinate object is established according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

2. The method for detecting a master-slave relationship between objects in an image according to claim 1, characterized in that: After obtaining pixel information of the image to be detected and constructing a canvas of equal resolution according to the pixel information, the method further includes: Setting the pixel value of each pixel in the canvas to a preset pixel value; Correspondingly, mapping the region information of the subject object to the canvas includes: Different pixel values ​​are assigned to each main object respectively, and the correlation relationship between the main object and the pixel value is obtained; According to the area information of the main object and the association relationship between the main object and the pixel value, the pixel value of each pixel in the canvas is adjusted.

3. The method for detecting a master-slave relationship between objects in an image according to claim 2, characterized in that: According to the region information of the main object and the association relationship between the main object and the pixel value, the pixel value of each pixel in the canvas is adjusted, including: Determining a pixel area corresponding to the main object in the canvas according to the area information of the main object; According to the association relationship between the main object and the pixel value, the pixel value of each pixel in the pixel area is set to the pixel value associated with the main object.

4. The method for detecting a master-slave relationship between objects in an image according to claim 2, characterized in that: According to the mapping result of the main object and the mapping result of the subordinate object in the canvas, a master-slave relationship between the main object and the subordinate object is established, including: Determine the target subject object according to the pixel value of each pixel in the mapping result of the subordinate object; It is determined that there is a master-slave relationship between the target principal object and the subordinate object.

5. The method for detecting a master-slave relationship between objects in an image according to claim 4, characterized in that: Determining the target main object according to the pixel value of each pixel in the mapping result of the subordinate object includes: Counting the number of pixels of each pixel value in the mapping result of the subordinate object; Determine the pixel value with the largest number of pixels as the target pixel value; The target subject object is determined according to the target pixel value and the association relationship between the subject object and the pixel value.

6. The method for detecting a master-slave relationship between objects in an image according to claim 1, characterized in that: Identifying the region information of at least one main object in the image to be detected includes: Inputting the image to be detected into a pre-trained first instance segmentation model, and determining region information of at least one main object in the image to be detected according to an output result of the first instance segmentation model; Accordingly, identifying the region information of at least one subordinate object in the image to be detected includes: The image to be detected is input into a pre-trained second instance segmentation model, and region information of at least one subordinate object in the image to be detected is determined according to an output result of the second instance segmentation model.

7. The method for detecting a master-slave relationship between objects in an image according to claim 6, characterized in that: The first instance segmentation model and the second instance segmentation model respectively adopt Yolov5 instance segmentation model.

8. A device for detecting a master-slave relationship between objects in an image, characterized in that: The device comprises: A canvas construction module, used to obtain pixel information of the image to be detected and construct a canvas of equal resolution according to the pixel information; A subject recognition and mapping module, used to recognize the region information of at least one subject object in the image to be detected, and map the region information of the subject object into the canvas; A subordinate identification and mapping module, used for identifying the region information of at least one subordinate object in the image to be detected, and mapping the region information of the subordinate object into the canvas; The master-slave relationship establishment module is used to establish a master-slave relationship between the main object and the subordinate object according to the mapping result of the main object and the mapping result of the subordinate object in the canvas.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method for detecting a master-slave relationship between objects in an image as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method for detecting a master-slave relationship between objects in an image according to any one of claims 1 to 7 are implemented.

Images