Object recognition method and device

Abstract

The present invention provides an object recognition method and device, the object recognition method comprising: obtaining an image to be recognized; processing the image to be recognized using a preset image segmentation method to obtain a first prediction result; based on the first prediction result, obtaining a second prediction result by finding an optimal solution to the objective function of a preset optimization model, wherein the objective function of the preset optimization model aims to maximize the sum of the probability thermal forces of multiple corner points of the target object recognition frame, the scene category of the target object, and the semantics of the target object; and obtaining target object recognition information based on the second prediction result. The method and device of the present invention optimize the recognition results using a preset optimization model, solve the problems of incompatibility of general object detection and low precision recall of general object detection recognition, and can use the recognition results as analysis data to support further applications such as design and recall.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an object recognition method and device. Background Art

[0002] At present, the technology of using artificial intelligence to recognize objects in images has been tried and applied in various fields, and most of them achieve the purpose of identifying objects by obtaining recognition frames.

[0003] In various fields, in addition to obtaining the recognition frame of an object, the need to obtain more advanced semantics of the object is becoming more and more urgent. For example, in the real estate field, in addition to recognizing the layout of the house shown in the floor plan, if the furniture, layout relationship and more advanced semantics can be accurately recognized, the internal situation of the house can be more comprehensively understood, and the recognition results can be used as analysis data to support more in-depth home decoration, design, recall and other applications.

[0004] Therefore, there is a need for an object recognition method and device that can provide more advanced semantics to overcome the above problems.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore it may contain information that does not constitute the prior art known to those skilled in the art. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention provides an object recognition method and device, which further screens the recognized objects by designing an optimization model, and can not only obtain the position and semantics of the object, but also obtain high-level semantic information such as the scene, layout relationship and size of the object.

[0007] The present invention provides an object recognition method, comprising: acquiring an image to be recognized; processing the image to be recognized using a preset image segmentation method to obtain a first prediction result, wherein the first prediction result includes a target object recognition frame and multiple corner points of the target object recognition frame, target object semantics, and a scene category; based on the first prediction result, obtaining a second prediction result by finding an optimal solution to an objective function of a preset optimization model, wherein the objective function of the preset optimization model aims to maximize the probability thermal sum of multiple corner points of the target object recognition frame, the scene category of the target object, and the target object semantics; and obtaining target object recognition information based on the second prediction result.

[0008] According to an embodiment of the present invention, the preset optimization model is an integer programming model, and its objective function is expressed by the following formula:

[0009]

[0010] in, Represent the probability heat of multiple corner points of the target object recognition box; represents the probability heat sum of the scene category of the target object and the semantics of the target object; where P(j) represents the target object recognition box containing corner point j; W P represents the global hyperparameters; represents the weight of P(j); S(s,i) represents the scene containing the scene category and the semantics of the target object; W Icon A global hyperparameter representing the semantics of the target object; The probability heat representing the semantics of the target object; W Scene A global hyperparameter representing the scene category; Represents the probability heat of the scene category where the target object is located.

[0011] According to an embodiment of the present invention, the constraint conditions of the preset optimization model include one or more of the following:

[0012] Nearest neighbor constraint: (P(i)+P(k))≤1, which means that the overlap rate between two different target object recognition frames P(i) and P(k) cannot be greater than a first preset threshold;

[0013] One-hot constraint: It means that the target object recognition box P(p) can only take one target object semantics I p (i), where Set icon Indicates the semantics of the preset target object;

[0014] Mutually exclusive constraint: (I(m)+I(k))≤1, which indicates that the target object semantics I(m) and I(k) of non-stacked target objects whose overlap rate is greater than the first preset threshold are mutually exclusive;

[0015] Master-slave constraints: It means that the semantics I(Sub) of the stackable target objects whose overlap rate is greater than the first preset threshold is less than the sum of the semantics I(main) of all intersecting main target objects, where Set icon Indicates the semantics of the preset target object;

[0016] Quantity constraints: It represents the scene type S icon Number of target objects that appear in (s) scene is less than the second preset threshold, where Set scene Indicates the preset scene category.

[0017] According to one embodiment of the present invention, the object recognition method also includes: performing data filtering on the first prediction result; wherein, in the step of obtaining the second prediction result by finding the optimal solution to the objective function of a preset optimization model based on the first prediction result, the first prediction result is subjected to the data filtering process.

[0018] According to an embodiment of the present invention, the first prediction result also includes the target object size, and data filtering of the first prediction result includes: filtering out the target object semantics or target object recognition frame whose difference between the target object size and a preset size is greater than a third preset threshold.

[0019] According to one embodiment of the present invention, the first prediction result also includes a partition heat map and an attachment heat map, and the size of the target object is obtained by the following steps: using OCR to detect the annotation size on the image to be identified to obtain a first scaling ratio; using the partition heat map and the threading method to obtain the partition scaling ratio; using the attachment heat map to obtain the attachment scaling ratio; using a preset algorithm to merge the first scaling ratio, the partition scaling ratio and the attachment scaling ratio to obtain a second scaling ratio; and calculating the target object size based on the second scaling ratio.

[0020] According to an embodiment of the present invention, the preset algorithm calculates the probability density distribution of the first scaling ratio, the partition scaling ratio and the attachment scaling ratio, and uses the scaling ratio with the highest probability as the second scaling ratio.

[0021] According to an embodiment of the present invention, the preset algorithm is to take the median of the first scaling ratio, the partition scaling ratio and the attachment scaling ratio, and use it as the second scaling ratio.

[0022] According to one embodiment of the present invention, the first prediction result also includes a heat map of the target object, and data filtering of the first prediction result also includes: calculating the pixel segmentation probability statistics of the target object based on the target object heat map, and filtering out pixels whose pixel segmentation probability statistics of the target object are less than a first preset probability.

[0023] According to an embodiment of the present invention, the pixel segmentation probability statistics of the target object are obtained by calculating one or more of the following values: a mean value, a maximum value, and a median value of the pixel segmentation probability of the target object.

[0024] According to one embodiment of the present invention, the first prediction result also includes a scene layout heat map, and data filtering of the first prediction result also includes: based on the scene layout heat map, calculating the pixel segmentation probability statistics of the scene layout where the target object recognition box is located, and filtering out pixels whose pixel segmentation probability statistics of the scene layout where the target object recognition box is located are less than a second preset probability.

[0025] According to one embodiment of the present invention, the pixel segmentation probability statistics of the scene layout where the target object recognition frame is located are obtained by calculating one or more of the following values: the mean, maximum, and median of the pixel segmentation probability of the scene layout where the target object recognition frame is located.

[0026] According to an embodiment of the present invention, the target object recognition information includes one or more of the following: scene information, size information, and combination relationship information.

[0027] According to an embodiment of the present invention, the target object recognition frame is a rectangular frame.

[0028] According to an embodiment of the present invention, the rectangular frame is obtained by two corner points located at diagonal positions.

[0029] According to another aspect of the present invention, there is also provided an object recognition device, comprising: an acquisition module, configured to: acquire an image to be recognized; an image segmentation module, configured to: process the image to be recognized using a preset image segmentation method to obtain a first prediction result, wherein the first prediction result includes a target object recognition frame and multiple corner points of the target object recognition frame, target object semantics and scene category; an optimization module, configured to: based on the first prediction result, obtain a second prediction result by finding an optimal solution to an objective function of a preset optimization model, wherein the objective function of the preset optimization model aims to maximize the probabilistic thermal sum of multiple corner points of the target object recognition frame, the scene category of the target object and the semantics of the target object; and an output module, configured to: obtain target object recognition information based on the second prediction result.

[0030] According to one embodiment of the present invention, the object recognition device also includes: a filtering module, which is configured to: perform data filtering on the first prediction result, wherein the optimization module is configured to: obtain the second prediction result by finding the optimal solution to the objective function of a preset optimization model based on the first prediction result after the data filtering.

[0031] According to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the object recognition method described above when executing the program.

[0032] According to another aspect of the present invention, there is further provided a computer-readable storage medium having a computer program stored thereon, which implements the steps of the object recognition method described above when the program is executed by a processor.

[0033] According to another aspect of the present invention, a computer program product is further provided, comprising computer instructions, which implement the steps of the object recognition method described above when executed by a processor.

[0034] The object recognition method and device provided by the present invention utilize a preset optimization model to optimize the recognition results, thereby solving the problems of incompatibility of general object detection and low accuracy and recall of general object detection recognition, and can utilize the recognition results as analysis data to support further applications such as design and recall. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other features of the present invention will be described in detail below with reference to specific exemplary embodiments shown in the accompanying drawings, which are given below only by way of illustration and therefore do not limit the present invention, in which:

[0036] Figure 1 An exemplary system architecture that is applicable to a specific embodiment of the object recognition method of the present invention is shown.

[0037] Figure 2 A flow chart of an object recognition method according to an embodiment of the present invention is shown.

[0038] Figure 3 A logical schematic diagram of an object recognition method according to an embodiment of the present invention is shown.

[0039] Figure 4 A flow chart of an object recognition method according to another embodiment of the present invention is shown.

[0040] Figure 5 A schematic diagram of constructing a rectangular frame according to an embodiment of the present invention is shown.

[0041] Figure 6 A schematic diagram showing the use of a threading method on a wall thermal map according to an embodiment of the present invention.

[0042] Figure 7 A thermal diagram of a door body according to an embodiment of the present invention is shown.

[0043] Figure 8A furniture heat map according to an embodiment of the present invention is shown.

[0044] Fig. 9 A partition heat map according to an embodiment of the present invention is shown.

[0045] Fig.10 A schematic structural diagram of an object recognition device according to an embodiment of the present invention is shown.

[0046] Fig.11 A schematic structural diagram of an object recognition device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] The present invention is described in detail below through specific embodiments so that those skilled in the art can easily implement the present invention according to the contents disclosed in this specification. The embodiments described below are only some embodiments of the present invention, not all. Based on the embodiments described in this specification, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments in this specification and the features in the embodiments can be combined with each other without conflict.

[0048] The terms used herein are only for the purpose of illustrating specific embodiments and are not intended to limit the present invention. Unless the context otherwise specifically indicates, the singular forms "one", "a kind of" and "the" used herein also include the plural forms "multiple", "multiple" and "the". The words "first", "second" and the like used herein are only used to distinguish different features, steps, operations, elements and / or parts, etc., and neither represent any specific technical meaning nor the necessary logical order between them. The term "multiple" used herein may refer to two or more, and the term "at least one" may refer to one, two or more. For any feature, step, operation, element and / or part mentioned herein, it is generally understood to be one or more if the context is not otherwise specifically indicated. It should also be understood that the words "include" and / or "comprising" used herein refer to the presence of the features, steps, operations, elements and / or parts, but do not exclude the presence or increase of one or more other features, steps, operations, elements, parts and / or their collections. The term "and / or" used herein includes any or all combinations of one or more related listed items. The element suffixes “module” and “unit” herein are only used for convenience of description and, thus, can be used interchangeably without having any distinct meanings or functions.

[0049] When the prior art related to the description of the present invention is obvious to those skilled in the art, its detailed description will be omitted. It should also be understood that the description of each embodiment in this specification focuses on emphasizing the differences between the embodiments, and the same or similar aspects between the embodiments can be referenced to each other. For the sake of brevity, this specification does not repeat them one by one.

[0050] like Figure 1 As shown schematically, an exemplary system architecture 100 that can be applied to a specific embodiment of the object recognition method of the present invention is shown. The system architecture 100 may include terminal devices 101, 102, 103, as well as a network 104 and a server 105. The network 104 is used to provide communication between the terminal devices 101, 102, 103 and the server 105, which may include various connection types, such as wired, wireless communication or optical fiber cable.

[0051] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104. Various communication client applications can be installed on terminal devices 101, 102, 103, such as image and video shooting applications, text input applications, web browser applications, professional field application software, search applications, instant messaging tools, email clients, social platform software, etc.

[0052] In a specific implementation, the terminal devices 101, 102, 103 can be implemented as hardware or software according to actual needs. When the terminal devices 101, 102, 103 are implemented as hardware, they can be various electronic devices with a (touch) display screen and support various inputs such as voice and text, including but not limited to personal computers (including laptops and desktop computers), tablet computers, smart phones, car terminals, e-book readers, video players, etc. When the terminal devices 101, 102, 103 are implemented as software, they can be installed in a suitable electronic device and implemented as multiple software or software modules (for example, to provide distributed services), or they can be implemented as a single software or software module. It should be understood that Figure 1 The examples of terminal devices 101, 102, and 103 described in and above are merely examples and should not be understood as specific limitations.

[0053] The server 105 may be a server that provides various services, such as a background server that provides analysis, response and support for various information input by the terminal devices 101, 102, 103, such as control signals, voice or text information. The background server may analyze and process the received control signals, voice or target text information, and feed back the processing results to the terminal devices 101, 102, 103 through the network 104.

[0054] In a specific implementation, the server 105 can be implemented as hardware or software according to actual needs. When the server 105 is implemented as hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is implemented as software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. It should be understood that Figure 1 The server 105 examples described in and above are merely examples and should not be construed as being specifically limited.

[0055] It should be noted that the object recognition method provided in the embodiment of the present application can be executed by the terminal devices 101, 102, 103, can also be executed by the server 105, or can be executed by the terminal devices 101, 102, 103 and the server 105 in cooperation. Accordingly, the object recognition device can be set in the terminal devices 101, 102, 103, can also be set in the server 105, can also be set in the terminal devices 101, 102, 103 and the server 105.

[0056] It is understandable that when the object recognition method provided in the embodiment of the present application is executed by the terminal devices 101 , 102 , and 103 , the above-mentioned system architecture 100 may not include the network 104 and the server 105 .

[0057] It should be understood that Figure 1 The number and types of terminal devices, networks and servers in the embodiment are only for illustration. In a specific implementation, any number and types of terminal devices, networks and servers may be provided according to actual needs.

[0058] like Figure 2 As shown, the present invention provides an object recognition method 1000. Specifically, the method 1000 includes obtaining an image to be recognized at S1100; processing the image to be recognized using a preset image segmentation method at S1200 to obtain a first prediction result; obtaining a second prediction result based on the first prediction result by finding an optimal solution for an objective function of a preset optimization model at S1300; and obtaining target object recognition information based on the second prediction result at S1400.

[0059] Specifically, method 1000 Figure 2At step S1100 shown in , an image to be identified is obtained. It should be noted that the present invention does not limit the image to be identified to be an image of a specific field or a specific type. On the contrary, the image to be identified described in the present invention may be an image containing a target object to be identified in various fields, such as a house containing furniture, a shelf with goods, an exhibition with booths of various brands, etc., in which the same or different objects are placed in different areas of a scene. Through the technical solution of the present invention, not only can the target object be accurately identified, but also more high-level semantics can be obtained, such as the position / area of ​​the target object in the scene, the association relationship between the target objects, and the actual size of the target object. In order to better describe the technical solution of the present invention and facilitate the understanding of the reader, the following will take the real estate field as an example for example analysis. In this field, the image to be identified can be a floor plan, and the target object to be identified can be various types of furniture.

[0060] It should also be noted that the floor plan described in the embodiments of the present invention may refer to floor plans of three-dimensional space or three-dimensional virtual space involved in various fields such as houses, gymnasiums, parks, office buildings, shopping streets, etc. For the sake of convenience, in some specific embodiments of this specification, a house for traditional residential purposes will be taken as an example to explain the technical solution of the present invention in detail, but it is not intended to limit the scope of protection of the present invention.

[0061] Subsequently, method 1000 Figure 2 At step S1200 shown, the image to be identified is processed using a preset image segmentation method to obtain a first prediction result, which includes a target object identification frame and multiple corner points of the target object identification frame, target object semantics and scene category. As known to those skilled in the art, image segmentation is a technique and process for dividing an image into several specific regions with unique properties and proposing targets of interest. It is a key step from image processing to image analysis. Image segmentation has been widely used in many fields, especially in image recognition and object detection. The preset image segmentation method described in the present invention can use algorithms known to those skilled in the art, such as Faster RCNN, RetinaNet, Mask RCNN, etc., and then train the image segmentation model according to a preset training method, so that the image to be identified can be input into the image segmentation model to output the recognition results of the image to be identified in multiple channels. It should be noted that the preset training method can be any method known to those skilled in the art, and the present invention is not limited to this.

[0062] Further references Figure 3 , which shows a logical schematic diagram of an object recognition method according to an embodiment of the present invention. Figure 3As shown, this embodiment takes the real estate field as an example, and inputs the two-dimensional floor plan stored in the form of a grid on the far left of the figure into the image segmentation model constructed by the deep neural network (DNN) on the right, and can output prediction results such as corner point heat map, wall heat map, room / partition heat map, door heat map, partition category, furniture corner point (i.e., identification box corner point), furniture identification box, furniture semantics, furniture heat map, etc. This step is in Figure 3 As is known to those skilled in the art, a heat map is an image obtained by highlighting each area in a target image using different display colors according to the significance probability value of each area. Thus, images showing different corners, walls, rooms / partitions, and doors can be obtained through the corner heat map, wall heat map, room / partition heat map, and door heat map. It should be noted that Figure 3 For the purpose of illustration, only five prediction results of room, furniture, wall, corner point and furniture semantic elements are shown, but this does not limit the present invention to other prediction results.

[0063] Taking the real estate field as an example, deep learning can be used through the above image segmentation model to predict the pixel segmentation probability of the furniture recognition box corner points, walls, partitions, furniture, etc. of the input floor plan through the channels of their respective categories, that is, to predict the corresponding probability of each pixel in the above different channels.

[0064] In a preferred embodiment of the present invention, the furniture recognition frame may be a rectangular frame, and the furniture may be framed therein when being recognized. The combined furniture may be obtained from a plurality of rectangular frames, and in this case, the furniture corner points may be four types of corner points of the rectangular frame, namely, the four corners of the rectangle, namely, the upper left, lower right, lower left and upper right. In other embodiments of the present invention, the furniture recognition frame may also be other shapes. For example, a polygonal recognition frame of irregular furniture (such as a U-shape) may be obtained using a polygon fitting function, and accordingly, the furniture corner points may be a plurality of corner points of a polygon. It should be noted that the furniture described here may correspond to object recognition frames in various fields including the real estate field, and thus the furniture corner points, furniture semantics and furniture thermal maps may correspond to target object recognition frame corner points, target object semantics and target object thermal maps in various fields. Among them, the semantics of furniture in the real estate field may include: dining table, cooking table, toilet, entrance door area, bed, special, stairs, pipe area, sofa, chair, tatami, refrigerator, bedside table, sink, bathtub, TV cabinet, coffee table, desk, vegetable sink, gas stove, air conditioner, cabinet, washing machine, shower, etc.

[0065] Taking the real estate field as an example, the wall may include a thick wall, a common wall, a fence, etc. It should be noted that the wall described here may correspond to a partition in various fields including the real estate field, and thus the wall heat map may correspond to a partition heat map in various fields.

[0066] Taking the real estate sector as an example, the room categories in the floor plan may include: living room, kitchen, bedroom, bathroom, shower area, foyer, balcony, storage room, corridor, dining room, bay window area, etc. It should be noted that the rooms described here may correspond to scene layouts in various fields including the real estate sector, and thus the room heat map and room categories described here may correspond to scene layout heat maps and scene categories in various fields.

[0067] Subsequently, method 1000 Figure 2 At step S1300 shown, based on the first prediction result, the second prediction result is obtained by finding the optimal solution to the objective function of the preset optimization model, wherein the objective function of the preset optimization model aims to maximize the sum of the probability heat of multiple corner points of the target object recognition frame, the scene category of the target object, and the semantics of the target object. As known to those skilled in the art, the probability heat can represent the significance probability value of each area in the image, and the probability heat of different image segmentation channels such as the multiple corner points of the target object recognition frame, the scene category of the target object, and the semantics of the target object can be calculated and obtained respectively.

[0068] Further references Figure 3 In a preferred embodiment of the present invention, as also in Figure 3 As shown in the "Prediction Phase", the decision variables (Proposal Variables) can be first defined for the preset optimization model. These decision variables are 0-1 variables, that is, these decision variables take values ​​0 or 1, where 0 represents discard and 1 represents selection, as follows:

[0069] a) Junction: The corner points that constitute the object recognition frame. In the above-mentioned embodiment in the real estate field, they are the corner points that constitute the rectangle. They can be expressed as J(j)∈{0,1}, where J represents a set of corner points and j represents a corner point element in the set;

[0070] b) Primitive: the recognition box of the object, which is a rectangular box in the above-mentioned embodiment of the real estate field, and can be represented as P(p)∈{0,1}, where P represents a set of recognition boxes and p represents a recognition box element in the set;

[0071] c) Object semantics (Icon): semantic representation of an object, which in the above-mentioned embodiment of the real estate field is furniture semantics (such as a chair), which can be represented as I(i)∈{0,1}, where I represents a set of object semantics and i represents an object semantic element in the set;

[0072] d) Objects in the scene (Scene / room): The scene representation of objects, in the above-mentioned real estate field embodiment, is the furniture in the partition (such as the chairs in the living room), which can be represented as S(s)∈{0,1}, where S represents the set of scene representations and s represents the scene representation element in the set.

[0073] Based on the above 0-1 decision variables, an integer programming optimization model can be constructed and Figure 3 The “optimization phase” shown on the right side optimizes the data. The objective function (Objective) of the preset optimization model of the present invention can be expressed by the following formula (1):

[0074]

[0075] in, Represents the probability heat of multiple corner points of the target object recognition box, that is, the regional heat summary of the corner points. Taking the real estate field as an example, this part of the optimization model is intended to encourage as many weighted recognition box corner points as possible, which helps to reward the separation of multiple sticky furniture. More specifically, in the formula, P(j) represents the target object recognition box containing the recognition box corner point j, which is a 0-1 decision variable; W P represents the global hyperparameters; The weight representing P(j) is equal to the probability heat of the corner point j of the recognition box. Preferably, the multiple corner points of the target object recognition box in the objective function can be two corner points of the rectangular box at diagonal positions.

[0076] in, represents the probability heat sum of the scene category of the target object and the semantics of the target object; S(s,i) represents the scene containing the scene category and the semantics of the target object, which is a 0-1 decision variable, where s represents the scene category and i represents the semantics of the target object; W Icon A global hyperparameter representing the semantics of the target object; The probability heat representing the semantics of the target object; W Scene A global hyperparameter representing the scene category; Represents the probability heat of the scene category where the target object is located.

[0077] Further references Figure 3 In the “optimization phase” shown on the right side, in the above embodiment of the present invention, the optimization model shown in formula (1) can also set constraints from the following three aspects:

[0078] (1) Geometric figures:

[0079] Nearest neighbor constraint:

[0080] (P(i)+P(k))≤1 (2)

[0081] The above formula (2) indicates that the overlap rate (IoU) between two different target object recognition frames P(i) and P(k) cannot be greater than the first preset threshold, where P(i) and P(k) are both 0-1 decision variables, where i and k respectively represent the recognition frame elements of two different target object recognition frame sets, where the first preset threshold can be set according to actual conditions or experience, and the present invention does not limit its specific value. Specifically, when the overlap between two recognition frames is high, it may mean that they have repeatedly recognized the same target object. Of course, it may not be the case. Therefore, the data that does not meet this constraint can be further judged by the mutual exclusion constraint and master-slave constraint in the object semantics of (2) below.

[0082] (2) Object semantics:

[0083] a) One-hot constraint:

[0084]

[0085] The above formula (3) indicates that the target object recognition box P(p) can only take one target object semantics I p (i), that is, a target object can only have one semantics, where P(p) and I p (i) are all 0-1 decision variables, Set icon Can express the preset semantics of target objects;

[0086] b) Exclusive constraint:

[0087] (I(m)+I(k))≤1 (4)

[0088] The above formula (4) indicates that the target object semantics I(m) and I(k) of non-stacked target objects whose overlap rate is greater than the first preset threshold are mutually exclusive, where I(m) and I(k) are both 0-1 decision variables, and m and k respectively represent the target object semantic elements in two different target object semantic sets. Specifically, non-stacked target objects refer to target objects that generally do not overlap in a two-dimensional image, such as beds and cabinets in the real estate field. Objects like these cannot overlap, but for combined furniture such as chairs and tables, gas stoves and cabinets, sinks and cabinets, etc., they are not mutually exclusive;

[0089] c) Master-slave constraint (Combination):

[0090]

[0091] The above formula (5) indicates that the semantics of the secondary target object I(Sub) of the stackable target object whose overlap rate is greater than the first preset threshold is less than the sum of the semantics of all intersecting main target objects I(main), that is, there must be an overlapping relationship between some objects, where I(Sub) and I(main) are both 0-1 decision variables, Set icon It can represent the preset target object semantics. For example, the sink and the cabinet must overlap in general. The sink can be used as the secondary furniture semantics (i.e., the secondary target object semantics I(Sub)), and the cabinet can be used as the main furniture semantics (i.e., the main target object semantics I(main)). The sink can only exist when the cabinet exists. That is, the above formula (5) can indicate that the secondary target object semantics can only exist when the main target object semantics exists. For the multiple main target object semantics corresponding to a certain secondary target object semantics that may be obtained by this constraint, other constraints can be used to effectively exclude and filter them.

[0092] (3)Scene category:

[0093] Amount constraint:

[0094]

[0095] The above formula (6) represents a specific scene type S icon The number of specific target objects appearing in (s) is less than the second preset threshold Number scene , where S icon (s) is a 0-1 decision variable, Set scene It can represent a preset scene category, wherein the second preset threshold can be set according to actual conditions or experience, and the present invention does not limit its specific value. That is to say, only a certain type of object is allowed to appear in a certain scene type, and there is a limit on the number of objects that can appear. For example, cabinets, refrigerators, gas stoves, and vegetable sinks cannot appear in toilets; toilets and showers cannot appear in kitchens; beds cannot appear in toilets and kitchens; a toilet can have at most one toilet, one shower, and one bathtub; a bedroom can have at most one bed and one TV; a living room can have at most one TV and one dining table; a unit can have at most one refrigerator and one gas stove, and so on.

[0096] In other embodiments of the present invention, those skilled in the art may also construct the objective function and constraint conditions of the preset optimization model to maximize the sum of multiple corner points of the target object recognition box, the scene category of the position of the target object in the scene layout heat map, and the probability heat of the target object semantics according to the characteristics and actual conditions of each field. The above formulas (1) to (6) are only examples of the technical solution of the present invention, and are not limitations of the present invention.

[0097] Subsequently, method 1000 Figure 2 At step S1400 shown, target object recognition information is obtained based on the second prediction result. Specifically, through the optimization model in the above embodiment of the present invention, target object recognition information containing more and more accurate high-level semantic information can be obtained, which may include one or more of the following: scene information, size information, and combination relationship information. Among them, scene information can express the scene position and category of the target object, such as a refrigerator in the kitchen; size information can express the size of the target object in the real or virtual world, such as the length, width and height of furniture in the physical world; combination relationship information can express the layout relationship of the target object, such as the combination relationship between tables and chairs, gas stoves and cabinets, sinks and cabinets, etc.

[0098] pass Figure 2 The object recognition method 1000 of an embodiment of the present invention shown in the figure takes the real estate field as an example. First, it can overcome the defect that the existing image detection model cannot consider the spatial relationship of furniture, such as cabinets cannot overlap, while the sink and the cabinet must overlap; secondly, with the help of the constraints of the apartment application scenario, the recall of reconstruction can be improved. Small object detection is one of the most classic adjustments in the field of image detection. Based on the optimization model, the weight of small objects can be controlled in a finer granularity; thirdly, with the help of the constraints of the apartment application scenario, the accuracy of reconstruction can be improved. For example, the actual situation that there is a high probability that there will be no gas stove in the toilet and there is a high probability that there is only one washing machine on the balcony is integrated into the optimization model.

[0099] Figure 4 An object recognition method 2000 according to another embodiment of the present invention is shown, and the object recognition method 2000 includes acquiring an image to be recognized at S2100; processing the image to be recognized using a preset image segmentation method at S2200 to obtain a first prediction result; filtering the first prediction result at S2300; obtaining a second prediction result by finding an optimal solution for an objective function of a preset optimization model based on the filtered first prediction result at S2400; and obtaining target object recognition information based on the second prediction result at S2500.

[0100] Specifically, method 2000 Figure 4 At steps S2100 to S2200 shown in FIG. Figure 2 The image to be identified is obtained in a similar manner as described in steps S1100 to S1200 in the embodiment and the image to be identified is processed using a preset image segmentation method to obtain a first prediction result.

[0101] Subsequently, Method 2000 Figure 4The first prediction result is subjected to data filtering at step S2300 shown. In one embodiment of the present invention, the target object recognition frame can be obtained based on the corner points of the recognition frame, or it can be directly obtained through a preset image segmentation model. Preferably, in an embodiment where the target object recognition frame is a rectangular frame, the rectangular frame can be obtained through two corner points at diagonal positions. Specifically, still taking the real estate field as an example, the four corner points of the rectangular recognition frame of a certain furniture can be first obtained using a preset image segmentation model, and then a rectangular frame can be constructed using the two upper left and lower right corner points (or the two lower left and upper right corner points) of the same furniture at diagonal positions, and this rectangular frame is used as the recognition frame of the furniture. The purpose of this is to avoid the appearance of furniture rectangular frames that span partitions, that is, to exclude furniture recognition frames that include walls in the middle, because in actual situations furniture cannot be truncated by walls. For further reference Figure 5 , which shows the construction of a rectangular recognition frame based on the furniture corner points based on the partition heat map, such as Figure 5 As shown, the recognition box can only be constrained within the same compartment and cannot be constructed across compartments.

[0102] In a preferred embodiment of the present invention, the first prediction result also includes the size of the target object, and data filtering of the first prediction result includes: filtering out the target object semantics or target object recognition frames whose difference between the target object size and the preset size is greater than a third preset threshold. Specifically, for a target object recognition frame, several semantics with a higher ranking can be obtained based on the probability of the target object semantics, and the value of the preset size can be determined in combination with the semantics. If the difference is too large, it indicates that the semantics is incorrect, and the semantics is filtered out, and then the judgment is made based on the next semantics, and so on, until the difference is within the preset range, which proves that the semantics is correct; optionally, the target object recognition frame that differs too much from the actual size of the target object can also be filtered out, that is, the recognition frame is excluded. Among them, the third preset threshold can be set according to actual conditions or experience, and the present invention does not limit its specific value.

[0103] In a more preferred embodiment of the present invention, the first prediction result also includes a partition heat map and an attachment heat map, and the size of the target object is obtained by the following steps: using optical character recognition (OCR) technology to detect the marked size on the image to be identified, and obtaining a first scaling ratio. Taking the real estate field as an example, the size of the house is generally marked on the house type map, and OCR can be used to identify these marked sizes, and then the first scaling ratio is calculated relative to the size of the house type map; using the partition heat map and the threading method to obtain the partition scaling ratio. Taking the real estate field as an example, further reference is made to Figure 6, which shows a schematic diagram of using the threading method on the wall heat map to obtain the wall scaling ratio. Specifically, first, N sampling points are randomly selected in the inner area of ​​the wall heat map, and N is preferably greater than 3. Then, horizontal and vertical rays are respectively emitted from the N sampling points and all encountered wall widths are recorded. Finally, the wall scaling ratio is calculated according to the general thickness of the actual wall. The attachment scaling ratio is obtained by using the attachment heat map. Taking the real estate field as an example, further reference is made to Figure 7 , which shows the door body thermal map, such as Figure 7 As shown, the door body is marked with a box in the floor plan, the width of the door body in the door body heat map is measured and the door body scaling ratio is calculated using the general width of the actual door body; the first scaling ratio, the partition scaling ratio and the accessory scaling ratio are combined using a preset algorithm to obtain a second scaling ratio; and the target object size is calculated based on the second scaling ratio. Optionally, the preset algorithm can be to calculate the probability density distribution of the first scaling ratio, the partition scaling ratio and the accessory scaling ratio, and take the scaling ratio with the largest probability as the second scaling ratio. Specifically, the three scaling ratios obtained in the above steps can be multiple, and then use these scaling ratios to calculate their probability density distributions, and take the scaling ratio with the largest probability to calculate the target object size. Optionally, the preset algorithm is to take the median of the first scaling ratio, the partition scaling ratio and the accessory scaling ratio, and take it as the second scaling ratio. Specifically, the three scaling ratios obtained in the above steps can be multiple, and then use these scaling ratios to calculate their median, and take the scaling ratio of the median to calculate the target object size.

[0104] In a preferred embodiment of the present invention, the first prediction result also includes a heat map of the target object, and filtering the data of the first prediction result may further include: calculating a pixel segmentation probability statistic of the target object based on the heat map of the target object, and filtering out pixels whose pixel segmentation probability statistic of the target object is less than a first preset probability. Figure 8 , which shows a furniture heat map, which can reflect the probability of each furniture channel. Preferably, the pixel segmentation probability statistics of the target object are obtained by calculating one or more of the following values: the mean, maximum, and median of the pixel segmentation probability of the target object. The pixel with a smaller probability indicates that the pixel is not the target object, so it needs to be excluded. Among them, the first preset probability can be set according to actual conditions or experience, and the present invention does not limit its specific value.

[0105] In another preferred embodiment of the present invention, the first prediction result also includes a scene layout heat map, and data filtering of the first prediction result further includes: calculating the pixel segmentation probability statistics of the scene layout at the location of the target object recognition frame based on the scene layout heat map, and filtering out pixels whose pixel segmentation probability statistics of the scene layout at the location of the target object recognition frame are less than a second preset probability. Fig. 9 , which shows a partition heat map, which can reflect the probability of each partition channel. Preferably, the pixel segmentation probability statistics of the scene layout at the location of the target object recognition frame are obtained by calculating one or more of the following values: the mean, maximum value, and median of the pixel segmentation probability of the scene layout at the location of the target object recognition frame. The pixel with a smaller probability indicates that the pixel is not the partition corresponding to the target object, so it needs to be excluded, for example, the toilet cannot appear in the kitchen, the dining table cannot appear in the toilet, etc. Among them, the second preset probability can be set according to actual conditions or experience, and the present invention does not limit its specific value. In the above embodiment of the present invention, the pixel segmentation probability of the scene layout at the location of the target object recognition frame is used for data filtering, which can avoid data interference caused by the situation that the apartment type is a large bay or other partition fusion. For example, if only the pixel segmentation probability of the partition where the furniture recognition frame is located is calculated, and the furniture happens to be in a bay apartment type, this will cause trouble for data filtering. Therefore, the embodiment of the present invention preferably only calculates the pixel segmentation probability of the location of the furniture in the partition, which can greatly improve the accuracy of data filtering.

[0106] Subsequently, Method 2000 Figure 4 At steps S2400 to S2500 shown in FIG. Figure 2 A similar method as described in steps S1300 to S1400 in the above method is based on the filtered first prediction result, and a second prediction result is obtained by finding an optimal solution to the objective function of a preset optimization model, and target object recognition information is obtained based on the second prediction result.

[0107] pass Figure 4The object recognition method 2000 of an embodiment of the present invention shown in the figure, taking the real estate field as an example, firstly, can improve the accuracy of object recognition, can support the recognition of at least 24 types of furniture, and the range of furniture types that can be recognized is expanded three times compared with the existing furniture recognition. Secondly, the furniture mutual exclusion assumption is broken, and compared with the existing furniture recall rate of 52%, the present invention can achieve a higher furniture recall rate of 73%. For example, the existing furniture reconstruction has a mutual exclusion assumption, that is, the furniture cannot overlap, but in fact, there is a possibility of overlap between cabinets and gas stoves, chairs and tables, etc., so the present invention breaks the above mutual exclusion constraint. Furthermore, the existing furniture reconstruction generally requires the use of four corner points of a rectangular recognition frame, while in the preferred embodiment described in the present invention, two corner points at diagonal positions can be used, thereby lowering the recall requirements. Furthermore, the existing model can only consider rectangular furniture, but most of the floor plans in CAD have a lot of non-rectangular furniture, such as L-shaped sofas, U-shaped cabinets, etc., and these existing models cannot handle them. In the present invention, the optimization model will convert non-rectangular furniture into modular furniture in the form of optimal segmentation. Furthermore, the present invention improves the existing prediction accuracy of 83% to 90% by adding advanced semantic constraints, achieving truly high-precision prediction. Specifically, since the furniture in the floor plan is often presented as a very simple wireframe, but the same frame has different semantics in different areas, such as the kitchen frame and the bedroom frame, one represents a cabinet and the other represents a table, and sometimes in the same compartment, it is even more difficult for the model to predict the category of the frame. In the present invention, due to the use of integer programming optimization model and the introduction of scene semantics, a large number of scene priors are used to constrain the category of the frame, which greatly improves the final prediction accuracy.

[0108] Based on the same inventive concept, Fig.10 The structure diagram of an object recognition device 3000 according to an embodiment of the present invention is shown. The device 3000 includes: an acquisition module 3100, which is configured to acquire an image to be recognized. In an embodiment of the present invention, the acquisition module 3100 can be configured to execute Figure 2 S1100 Figure 4 S2100 shown in FIG. 1 and in this specification Figure 2 S1100 Figure 4 The image segmentation module 3200 is configured to process the image to be identified by using a preset image segmentation method to obtain a first prediction result, wherein the first prediction result includes a target object recognition frame and multiple corner points of the target object recognition frame, target object semantics and scene category. In an embodiment of the present invention, the image segmentation module 3200 can be configured to perform Figure 2 S1200, Figure 4 S2200 shown in FIG. Figure 2 S1200, Figure 4 The step corresponding to S2200 in the above description; an optimization module 3300, which is configured to obtain a second prediction result by finding an optimal solution to the objective function of a preset optimization model based on the first prediction result, wherein the objective function of the preset optimization model aims to maximize the sum of the probability thermal forces of multiple corner points of the target object recognition frame, the scene category of the target object, and the semantics of the target object. In an embodiment of the present invention, the optimization module 3300 can be configured to execute Figure 2 S1300, Figure 4 S2400 shown in FIG. Figure 2 S1300, Figure 4 The step corresponding to S2400 in the above description; and an output module 3400, which is configured to obtain target object recognition information based on the second prediction result. In an embodiment of the present invention, the output module 3400 may be configured to execute Figure 2 S1400 Figure 4 S2500 shown in FIG. Figure 2 S1400 Figure 4 The corresponding steps for S2500 in .

[0109] Based on the same inventive concept, Fig.11 FIG. 4 is a schematic diagram showing a structure of an object recognition device 4000 according to another embodiment of the present invention. The device 4000 includes: an acquisition module 4100 configured to acquire an image to be recognized. In an embodiment of the present invention, the acquisition module 4100 may be configured to execute Figure 2 S1100 Figure 4 S2100 shown in FIG. 1 and in this specification Figure 2 S1100 Figure 4 The image segmentation module 4200 is configured to process the image to be identified by using a preset image segmentation method to obtain a first prediction result, wherein the first prediction result includes a target object recognition frame and multiple corner points of the target object recognition frame, target object semantics and scene category. In an embodiment of the present invention, the image segmentation module 4200 can be configured to perform Figure 2 S1200, Figure 4 S2200 shown in FIG. Figure 2 S1200, Figure 4 The step corresponding to S2200 in the above description; a filtering module 4300, which is configured to perform data filtering on the first prediction result. In an embodiment of the present invention, the data filtering module 4300 can be configured to perform Figure 4 S2300 shown in FIG. Figure 4The step corresponding to S2300 in FIG. 4; an optimization module 4400, which is configured to obtain a second prediction result by finding an optimal solution to the objective function of a preset optimization model based on the filtered first prediction result. In an embodiment of the present invention, the optimization module 4400 can be configured to perform Figure 2 S1300, Figure 4 S2400 shown in FIG. Figure 2 S1300, Figure 4 The step corresponding to S2400 in the above description; and an output module 4500, which is configured to obtain target object recognition information based on the second prediction result. In an embodiment of the present invention, the output module 4500 may be configured to execute Figure 2 S1400 Figure 4 S2500 shown in FIG. Figure 2 S1400 Figure 4 The corresponding steps for S2500 in .

[0110] Understandably, Fig.10 and 11 The structure shown is for illustration only, and the device may also include Fig.10 and 11 More or fewer modules or components shown in, or with Fig.10 and 11 Different configurations are shown.

[0111] In summary, the object recognition method and device described in the present invention adopts a two-stage modeling paradigm of deep learning and integer programming. Taking the real estate field as an example, DNN is used to predict key points and furniture segmentation as the input of the next stage model, and high-level semantic constraints and optimization are implemented based on the integer programming model to obtain the final vector furniture. With the help of the rich information obtained by the above embodiments of the present invention, we can reconstruct a complete furniture layout plan vector. In addition to the value of data reuse and migration itself, more importantly, standardized learning data is accumulated for the future use of models to imitate the layout ideas of designers.

[0112] In addition, the present application also provides a computer device. According to one embodiment of the present invention, the computer device may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the object recognition method described in this specification may be implemented.

[0113] In addition, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device can perform the steps of the object recognition method described in this specification.

[0114] In addition, the present application also provides a computer program product, including computer instructions, which, when executed by a processor, can implement the steps of the object recognition method described in this specification.

[0115] In particular, the embodiment process described above with reference to the flowcharts in the accompanying drawings can be implemented as a computer software program. For example, the embodiment disclosed in the specification of this application includes a computer program product, which includes a computer program carried on a computer-readable medium, the computer program includes program codes for executing the methods shown in the flowcharts in the accompanying drawings, and the computer program is executed by a processor to execute the method of the present application.

[0116] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: a computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0117] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0118] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0119] The flow chart and block diagram in the accompanying drawings exemplarily illustrate the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present application. Each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should be noted that in some alternative embodiments, the functions marked in the box can also occur in a different order from that shown in the accompanying drawings. For example, two boxes represented in sequence can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] The units or modules involved in the embodiments of the present application may be implemented by software or hardware. The above units or modules may also be arranged in a processor, for example, may be described as: a processor including an acquisition module, an image segmentation module, an optimization module, an output module, etc. The names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0121] All documents mentioned in this specification are cited in this application as references, just as if each document was fully cited in this specification as references.

[0122] In addition, it should be understood that after reading the above description of the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalent forms also fall within the protection scope of the present invention.

Claims

1. A method for object recognition, characterized in that: include: Obtain an image to be recognized; Processing the image to be identified by using a preset image segmentation method to obtain a first prediction result, wherein the first prediction result includes a target object recognition frame and multiple corner points of the target object recognition frame, target object semantics, and scene category; Based on the first prediction result, a second prediction result is obtained by finding an optimal solution to an objective function of a preset optimization model, wherein the objective function of the preset optimization model aims to maximize the sum of the probability thermal forces of multiple corner points of the target object recognition frame, the scene category of the target object, and the semantics of the target object; and Obtaining target object recognition information based on the second prediction result, The preset optimization model is an integer programming model, and its objective function is expressed by the following formula: in, Represent the probability heat of multiple corner points of the target object recognition box; represents the probability heat sum of the scene category of the target object and the semantics of the target object; where P(j) represents the target object recognition box containing corner point j; W P represents the global hyperparameters; represents the weight of P(j); S(s,i) represents the scene containing the scene category and the semantics of the target object; W Icon A global hyperparameter representing the semantics of the target object; The probability heat representing the semantics of the target object; W Scene A global hyperparameter representing the scene category; Represents the probability heat of the scene category where the target object is located.

2. The method according to claim 1, characterized in that The constraints of the preset optimization model include one or more of the following: Nearest neighbor constraint: (P(i)+P(k))≤1, which means that the overlap rate between two different target object recognition frames P(i) and P(k) cannot be greater than a first preset threshold; One-hot constraint: It means that the target object recognition box P(p) can only take one target object semantics I p (i), where Set icon Indicates the semantics of the preset target object; Mutually exclusive constraint: (I(m)+I(k))≤1, which indicates that the target object semantics I(m) and I(k) of non-stacked target objects whose overlap rate is greater than the first preset threshold are mutually exclusive; Master-slave constraints: It means that the semantics I(Sub) of the stackable target objects whose overlap rate is greater than the first preset threshold is less than the sum of the semantics I(main) of all intersecting main target objects, where Set icon Indicates the semantics of the preset target object; Quantity constraints: It represents the scene type S icon Number of target objects that appear in (s) scene is less than the second preset threshold, where Set scene Indicates the preset scene category.

3. The method according to claim 1, characterized in that The method further comprises: Performing data filtering on the first prediction result; Wherein, in the step of obtaining the second prediction result by finding the optimal solution to the objective function of the preset optimization model based on the first prediction result, the first prediction result is subjected to the data filtering process.

4. The method according to claim 3, characterized in that The first prediction result also includes the target object size, and data filtering of the first prediction result includes: filtering out target object semantics or target object recognition frames whose difference between the target object size and a preset size is greater than a third preset threshold.

5. The method according to claim 4, characterized in that The first prediction result also includes a partition heat map and an attachment heat map, and the target object size is obtained by the following steps: Using OCR to detect the size of the annotation on the image to be recognized, and obtain a first scaling ratio; Using the partition heat map and threading method to obtain the partition scaling ratio; Obtaining an attachment scaling ratio using the attachment heat map; Using a preset algorithm to combine the first scaling ratio, the partition scaling ratio and the attachment scaling ratio to obtain a second scaling ratio; as well as The target object size is calculated based on the second scaling factor.

6. The method according to claim 5, characterized in that The preset algorithm is to calculate the probability density distribution of the first scaling ratio, the partition scaling ratio and the attachment scaling ratio, and use the scaling ratio with the highest probability as the second scaling ratio.

7. The method according to claim 5, characterized in that The preset algorithm is to take the median of the first scaling ratio, the partition scaling ratio and the attachment scaling ratio, and use it as the second scaling ratio.

8. The method according to claim 3, characterized in that The first prediction result also includes a target object heat map, and filtering data on the first prediction result further includes: A pixel segmentation probability statistic of the target object is calculated based on the target object heat map, and pixels whose pixel segmentation probability statistic of the target object is less than a first preset probability are filtered out.

9. The method according to claim 8, characterized in that The pixel segmentation probability statistics of the target object are obtained by calculating one or more of the following values: the mean, maximum, and median of the pixel segmentation probability of the target object.

10. The method according to claim 3, characterized in that The first prediction result also includes a scene layout heat map, and filtering data on the first prediction result further includes: Based on the scene layout heat map, the pixel segmentation probability statistics of the scene layout where the target object recognition frame is located are calculated, and pixels whose pixel segmentation probability statistics of the scene layout where the target object recognition frame is located are less than a second preset probability are filtered out.

11. The method according to claim 10, characterized in that The pixel segmentation probability statistics of the scene layout where the target object recognition frame is located are obtained by calculating one or more of the following values: the mean, maximum, and median of the pixel segmentation probability of the scene layout where the target object recognition frame is located.

12. The method according to claim 1, characterized in that The target object recognition information includes one or more of the following: scene information, size information, and combination relationship information.

13. The method according to claim 1, characterized in that The target object recognition frame is a rectangular frame.

14. The method according to claim 13, characterized in that The rectangular frame is obtained by two corner points located at diagonal positions.

15. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 14 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 14 are implemented.

17. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 14 are implemented.

Images