Scene information identification method, model training method and related device

Abstract

The invention discloses a scene information identification method, a model training method and a related device. The method comprises the following steps: acquiring a scene image acquired from a target scene; performing semantic prediction on the scene image by using a semantic network of a scene information identification model to obtain a semantic prediction result of the scene image, the semantic prediction result comprising category information of different pixel points in the scene image, and the category information representing a target category to which the pixel points belong; the semantic prediction result is converted into scene information of the scene image through a matching network of the scene information recognition model, the scene information comprises position parameters of a plurality of targets in the scene image, and the parameter format of the position parameters is related to the attributes of the targets. Through the method, the scene applicability can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent perception technology, and in particular to a scene information recognition method, a model training method, and related devices. Background Art

[0002] Intelligent perception technology can automatically identify and classify specific environmental scenes in images, videos, or sensor data. However, existing intelligent perception technology generally detects and segments objects in scene images. The recognition results it outputs are often low-level pixel-level information, which requires complex post-processing to obtain scene information (such as fitting the corresponding scene information through multiple post-operations such as domain value judgment and logical association). This bottom-up approach, because it processes information at a very low level and has a high degree of freedom, leads to complex post-processing steps and poor scene applicability. Summary of the Invention

[0003] The main technical solution of this application is to provide a scene information recognition method, a model training method and related devices, which can improve the applicability of the scene.

[0004] To solve the above technical problems, a technical solution adopted in the present application is: to provide a scene information recognition method, the method comprising: obtaining a scene image obtained by collecting a target scene; performing semantic prediction on the scene image using a semantic network of a scene information recognition model to obtain a semantic prediction result of the scene image, wherein the semantic prediction result includes category information of different pixel points in the scene image, and the category information represents the target category to which the pixel points belong; and converting the semantic prediction result into scene information of the scene image using a matching network of the scene information recognition model, wherein the scene information includes position parameters of several targets in the scene image, and wherein the parameter format of the position parameters is related to the attributes of the targets.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: providing a training method for a scene information recognition model, the method comprising: obtaining a number of sample scene images, wherein the sample scene images are annotated with actual semantic results and actual scene information of the sample scene images, the actual semantic results include actual category information of different pixel points in the sample scene images, and the actual scene information includes actual position parameters of each element target in the sample scene image; using the semantic network of the scene information recognition model to perform semantic prediction on the sample scene images to obtain sample semantic prediction results of the scene images, wherein the sample semantic prediction results include predicted category information of different pixel points in the scene image; using the matching network of the scene information recognition model to convert the sample semantic prediction results into sample scene information of the sample scene images, wherein the sample scene information includes sample position parameters of several targets in the scene image; adjusting the semantic network and matching network of the scene information recognition model based on the difference between the actual semantic result and the sample semantic prediction result, and the difference between the actual scene information and the sample scene information.

[0006] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the above-mentioned scene information recognition method or the above-mentioned scene information recognition model training method.

[0007] To solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed to implement the above-mentioned scene information recognition method or the above-mentioned scene information recognition model training method.

[0008] The above scheme obtains a scene image obtained by collecting the target scene. The semantic network of the scene information recognition model is used to perform semantic prediction on the scene image to obtain the semantic prediction result of the scene image, and the matching network of the scene information recognition model is used to convert the semantic prediction result into the scene information of the scene image. Among them, the semantic prediction result includes the category information of different pixels in the scene image, and the category information represents the target category to which the pixel belongs. The scene information includes the position parameters of several targets in the scene image, and the parameter format of the position parameters is related to the attributes of the target. Therefore, the scene information in units of targets is directly obtained by using the scene information recognition model, that is, the direct recognition of target-level scene information is achieved in an end-to-end manner. Compared with pixel-level information, the amount of data of the recognition result can be effectively reduced, thereby avoiding complex post-operations and further improving the applicability of the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flow chart of an embodiment of a scene information recognition method provided by this application;

[0010] Figure 2 Provides a schematic diagram of the planar target description in the scene information recognition method for this application;

[0011] Figure 3 Provides a schematic diagram of stereoscopic target description in the scene information recognition method for this application;

[0012] Figure 4 This is a flowchart of a specific embodiment of scene information recognition model training in the scene information recognition method provided by this application;

[0013] Figure 5 This is a flow chart of an embodiment of a scene information recognition model training method provided by the present application;

[0014] Figure 6 This is a schematic diagram of the framework of an embodiment of the scene information recognition device of the present application;

[0015] Figure 7 This is a schematic diagram of the framework of an embodiment of the scene information recognition device of the present application;

[0016] Figure 8 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and effects of this application clearer and more specific, this application is further described in detail below with reference to the accompanying drawings and examples.

[0018] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. The term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects are in an "or" relationship. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of items. For example, at least one of A, B, and C can represent any one or more elements selected from the set consisting of A, B, and C.

[0019] See also Figure 1 , Figure 1 It is a flow chart of an embodiment of the scene information recognition method provided by this application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0020] Step S11: Acquire a scene image acquired by capturing the target scene.

[0021] The target scene in this article refers to the specific actual environment, situation, or background that is selected and focused on in practical applications or research problems. The collected scene images can be used to indicate the visual information in the target scene.

[0022] In a specific implementation, the target scene may be a traffic scene, and the corresponding scene image is a traffic scene image.

[0023] The method of acquiring scene images is flexible. It can be directly acquired through image acquisition equipment, or multiple frames of scene images can be extracted from the acquired scene video and the required scene images can be screened out from the multiple frames of scene images. There is no restriction here.

[0024] Step S12: Using the semantic network of the scene information recognition model to perform semantic prediction on the scene image, and obtain a semantic prediction result of the scene image.

[0025] Among them, the semantic prediction results include category information of different pixels in the scene image, and the category information represents the target category to which the pixel belongs.

[0026] In one embodiment, the semantic network of the scene information recognition model includes a semantic prediction subnetwork, and the semantic prediction subnetwork can be used to perform semantic prediction on the scene image to obtain a semantic prediction result of the scene image.

[0027] In another embodiment, the category information is the confidence level of the pixel belonging to each target category.

[0028] In a specific embodiment, the category information of different pixels in the scene image can be stored in a database or image structure. Taking the category information of different pixels stored in a heat map as an example, the semantic network of the scene information recognition model is used to output a heat map of size H×W×C, where H is the image height of the heat map, W is the image width of the heat map, and C is the number of all categories in the heat map (the number of target categories plus 1, 1 is the background category). The value of each pixel position in the heat map represents the confidence that the pixel is a certain target category. For example, the value of position (x, y) in the scene image is p i (x, y), p i (x, y) represents the pixel at position (x, y) which is the target C i confidence level.

[0029] In another embodiment, considering that semantic prediction of a scene image using a semantic network of a scene information recognition model may involve scaling of the scene image, the semantic prediction result may further include offset positions of different pixels, where the offset position indicates the position of the pixel in the scene image.

[0030] In a specific embodiment, the offset position can be stored in a database or image structure. For example, the scene image is predicted using a semantic network of a scene information recognition model to obtain an image of size H×W×2, where H is the image height, W is the image width, and 2 can represent the offset of each pixel point to each coordinate axis direction (x and y) of the vector from the scaling starting point.

[0031] In another specific embodiment, to uniformly handle the scale differences of the offset positions of different pixel points, the offset positions can be normalized so that their value range is uniformly mapped to the standardized interval [0, 1]. That is, the offset positions corresponding to different pixels in the scene image are normalized into normalized coordinates. Specifically, the original offset positions of different pixel points can be subjected to a linear or nonlinear scale transformation, and the final output is a standardized value between 0 and 1, i.e., the normalized offset position. The normalized offset position is then used as the semantic prediction result.

[0032] In another embodiment, taking into account the possibility that multiple instances (multiple targets) of the same category of targets may coexist in the scene image, the semantic prediction result also includes target information of different pixel points, wherein the target information is used to determine the pixel points belonging to the same target.

[0033] In one specific embodiment, target information can be stored in a database or image structure. For example, a semantic network of a scene information recognition model is used to predict a scene image, resulting in an image of size H×W×D, where H is the image height, W is the image width, and D is the dimension of the target information. Target information for the same target should be sufficiently close, while target information for different targets should be sufficiently far apart.

[0034] In yet another specific embodiment, the target information is an embedded feature.

[0035] For example, if D = 4 and the output image size is H × W × 4, 4 is the dimension of the embedded feature, that is, different pixels are mapped to a 4-dimensional continuous space vector. The embedded features of the same object should be close enough, and the embedded features of different objects should be far enough apart.

[0036] In another embodiment, the semantic prediction result further includes the offset positions of different pixels and target information of different pixels. The offset positions are used to indicate the positions of the pixels in the scene image, and the target information is used to determine the pixels belonging to the same target. For details, please refer to the relevant description in step S12 and will not be repeated here.

[0037] In another embodiment, the category information is the confidence that the pixel belongs to each target category. The semantic prediction result is a semantic feature map with semantic information. The size of the semantic feature map can be a scaled version of the scene image, and the offset position includes the offset of the vector from the pixel to the scaling starting point in each coordinate axis direction. The target information is an embedded feature. For detailed instructions, please refer to the relevant description in step S12 and will not be repeated here.

[0038] In another embodiment, the semantic network may include, in addition to the semantic prediction subnetwork, a feature extraction subnetwork and a feature fusion subnetwork. Specifically, the feature extraction subnetwork may be used to extract features from the scene image to obtain a multi-scale scene feature map. The feature fusion subnetwork may be used to fuse the multi-scale scene feature maps to obtain a fused feature map. The semantic prediction subnetwork may then be used to perform semantic prediction based on the fused feature map to obtain a semantic prediction result for the scene image.

[0039] In a specific embodiment, the semantic network can be a keypoint detection network or a segmentation network.

[0040] In another specific embodiment, a semantic network of a scene information recognition model is used to perform semantic prediction on a scene image to obtain a semantic feature map with semantic information, i.e., a semantic prediction result. Different pixels in the semantic feature map contain semantic information corresponding to different positions.

[0041] In another specific embodiment, the feature extraction subnetwork mainly extracts features from the preprocessed scene image, and the feature extraction subnetwork output contains features at multiple scales. Typical neural networks, such as ResNet-50, ResNet101, etc., can be used as backbone networks for feature extraction. For example, the size and numerical range of the scene image are first initialized, and then the processed scene image is input into the backbone network for convolutional feature extraction to obtain a feature map. It is denoted as {F1, F2, ...}. The features of each scale have different resolutions and numbers of channels, representing the low-level to high-level hierarchical features of the image. Taking ResNet-50 as the backbone network as an example, the feature outputs from stage2 to stage5 can be taken as the output of the feature extraction subnetwork.

[0042] In another specific embodiment, a feature fusion subnetwork is added after the feature extraction subnetwork, which mainly fuses the feature maps of multiple scales output by the feature extraction subnetwork. The feature fusion subnetwork structure adopts the feature pyramid network (FPN) structure, and outputs the convolution feature F∈R containing multi-scale information. H×W×N , where H is the image height, W is the image width, and N is the feature dimension.

[0043] In another specific embodiment, the semantic prediction subnetwork performs semantic prediction on the scene image based on the multi-scale feature map set obtained by the feature fusion subnetwork. The semantic prediction subnetwork can adopt a cascade hourglass structure.

[0044] Step S13: using the matching network of the scene information recognition model to convert the semantic prediction result into scene information of the scene image.

[0045] The scene information includes position parameters of several objects in the scene image, wherein the parameter format of the position parameters is related to the attributes of the objects.

[0046] The "several targets" in this article may refer to all targets that can be identified in the target scene, or may refer to targets identified in the scene image, which will not be elaborated here. For example, all targets that can be identified in the target scene are a, b, and c, and the targets identified in the scene image are a and b. The output scene information includes the position parameters of a, b, and c in the scene image. Among them, the position parameter of c that is not detected in the scene image is filled with zero value. For another example, the targets identified in the scene image are a and b. The output scene information includes the position parameters of a and b in the scene image.

[0047] The matching network in this paper can directly transform semantic prediction results into parameterized targets, that is, directly convert semantic prediction results into scene information in the scene image. The matching network can be composed of multiple standard convolutional layers and fully connected layers. The matching network can also include a regression subnetwork, which can be used to obtain the position parameters of several targets in the scene image.

[0048] In one embodiment, the plurality of objects are planar objects, and the position parameters of the planar objects are composed of coordinates of at least two first position points of the planar objects.

[0049] In a specific embodiment, when the planar target is a planar linear target, the at least two first location points include the starting point and the end point of the planar linear target. When the planar target is a planar area target, the at least two first location points include at least two vertices of the minimum circumscribed polygon of the planar area target.

[0050] In another specific embodiment, the planar linear target includes a line, and the position parameter of the line also includes a fitting parameter in a fitting formula of the line.

[0051] In another specific embodiment, the planar linear target includes an arrow, and the position parameters of the arrow include the coordinates of the starting point, the middle point, and the end point of the arrow.

[0052] In another specific embodiment, the position parameters of the planar area type target include the coordinates of each vertex of the minimum circumscribed polygon of the planar area type target, and the shape of the minimum circumscribed polygon of the planar area type target is related to the attributes of the planar area type target.

[0053] The above coordinates may refer to image coordinates in the scene image.

[0054] For example, if the target scene is a traffic scene, plane linear targets include at least one of lines and arrows, lines include lane lines, arrows include turn arrows, and plane area targets include at least one of zebra crossing areas, grid areas, diversion areas, and areas to be turned. For example, if the target scene is a traffic scene, plane linear targets include lane lines and turn arrows, and plane area targets include zebra crossing areas, grid areas, diversion areas, and areas to be turned.

[0055] The position parameters of the lane line can be expressed as (p s , p e , Θ) form, where p s , p e They are respectively represented as the starting coordinates and the ending coordinates of the lane line, θ is the fitting parameter in the lane line fitting formula, and the fitting formula can be a cubic polynomial. In addition, lane lines include vertical lane lines and horizontal lane lines, and the fitting formulas for different types of lane lines are also different. Specifically, for vertical lane lines, the corresponding fitting formula is x = θ0 + θ1y + θ2y 2 +θ3y 3 , that is, p s =(x s ,y s ), p e =(x e ,y e ), θ=(θ0,θ1,θ2,θ3). For the lateral lane line (such as the stop line), the corresponding fitting formula is y=θ0+θ1x+θ2x 2 +θ3x 3 , that is, p s =(x s ,y s ), p e =(x e ,y e), Θ=(θ0,θ1,θ2,θ3). In short, the position parameters of the lane line are expressed as (x s ,y s , x e ,y e , θ0, θ1, θ2, θ3), a set of position parameters represents a lane line, and there are 8 parameters corresponding to a lane line.

[0056] The position parameters of a turn arrow are expressed as (p1, p2, p3), where p1, p2, and p3 represent the coordinates of the start, middle, and end points of the turn arrow, respectively. The middle point can be the coordinates of the turning point or the midpoint of the arrow, and can be set based on actual conditions. This is not a limitation. In short, the position parameters of a turn arrow are expressed as (x1, y1, x2, y2, x3, y3). A set of position parameters represents a turn arrow, and a turn arrow has a total of six parameters.

[0057] A zebra crossing area can be represented by a closed convex hull polygon. This closed convex hull polygon is the minimum circumscribed polygon of the zebra crossing area. The position parameters of the zebra crossing area are expressed in the form of (p1, p2, p3, p4). Specifically, p1, p2, p3, and p4 represent the coordinates of the four vertices of the minimum circumscribed polygon of the zebra crossing area. In short, the position parameters of the zebra crossing area are expressed as (x1, y1, x2, y2, x3, y3, x4, y4). A set of position parameters represents a zebra crossing area, and a total of 8 parameters correspond to a zebra crossing area.

[0058] The position parameters of the grid area, diversion area, and waiting area are expressed in a similar way to the position parameters of the zebra crossing area. Specifically, the grid area is represented by the 4 vertex coordinates (x1, y1, x2, y2, x3, y3, x4, y4) of the smallest circumscribed polygon connected end to end. A set of position parameters represents a grid area, and a total of 8 parameters correspond to a grid area. The diversion area and the waiting area are represented by the 6 vertex coordinates of the smallest circumscribed polygon connected end to end. Specifically, they are represented by (x1, y1, x2, y2, x3, y3, x4, y4, x5, y4, x6, y6). A set of position parameters represents a diversion area or a waiting area, and a total of 12 parameters correspond to a diversion area or a waiting area.

[0059] It should be noted that the number of parameters of the actual position parameters of the zebra crossing area, grid area, diversion area and waiting area can be adjusted according to the number of vertices of the actual minimum circumscribed polygon, and is not limited here.

[0060] Combine Figure 2 To illustrate, Figure 2 This application provides a schematic diagram of the description of a planar target in a scene information recognition method. In FIG2a, there is an image of a linear target, and the first and last ends of the image are the starting point coordinates p and s and the end point coordinates p e , the parameter corresponding to the line segment connecting the beginning and the end of the figure is the fitting parameter Θ, so the position parameter corresponding to the line target is (p s , p e , Θ). Figure 2b shows the image of a plane area target, represented here by a minimum circumscribed quadrilateral. The vertex coordinates of this minimum circumscribed quadrilateral are p1, p2, p3, and p4, respectively. Therefore, the position parameters corresponding to this plane area target are (p1, p2, p3, p4). Figure 2c shows the image of an arrow type target. There are three arrows. The position parameters corresponding to the first arrow are (p1, p2, p3), the position parameters corresponding to the second arrow are (p4, p5, p6), and the position parameters corresponding to the third arrow are (p7, p8, p9).

[0061] In another embodiment, the plurality of objects are stereoscopic objects. Given that a stereoscopic object has a certain height within the target scene, its actual position on the ground and its position in the scene image may be inconsistent under a monocular vision imaging system. Therefore, the position parameters of the stereoscopic object may include ground projection position parameters and image position parameters. The ground projection position parameters are composed of the coordinates of at least two second position points of the projection area of ​​the stereoscopic object on the ground plane, and the image position parameters are composed of the coordinates of at least two third position points of the outline area of ​​the stereoscopic object in the scene image.

[0062] In a specific embodiment, the at least two second location points include at least two vertices of the minimum circumscribed polygon of the projection area. The at least two third location points include at least two vertices of the minimum circumscribed polygon of the contour area.

[0063] In another specific embodiment, the ground projection position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the projection area of ​​the stereoscopic target, the image position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the contour area of ​​the stereoscopic target, and the shape of the minimum circumscribed polygon corresponding to the stereoscopic target is related to the attributes of the stereoscopic target.

[0064] In another specific embodiment, the coordinates in the image position parameters may be image coordinates. The ground projection position parameters may be mapping coordinates that have undergone a geometric transformation, i.e., the ground projection position parameters may be mapping coordinates mapped to a two-dimensional space. For example, the ground projection position of a stereo object may be estimated by combining pre-calibrated parameters (such as camera internal and external parameters, ground equations) with dynamic reasoning to estimate the projection position of the stereo object on the ground plane, i.e., the ground projection position. For example, the ground projection position parameters may be estimated based on the image position of the stereo object in the scene image using pre-calibrated parameters.

[0065] For example, the target scene is a traffic scene, and the 3D objects include at least one of a green belt, a guardrail, a soundproof wall, a water barrier, a fence, and a channelized island. Taking the target scene as a traffic scene, and the 3D objects including a green belt, a guardrail, a soundproof wall, a water barrier, a fence, and a channelized island as an example, both the ground projection position parameter α and the image position parameter β can be represented by a closed convex hull polygon, that is, by a minimum circumscribed polygon.

[0066] The ground projection position parameter α and image position parameter β of the green belt and channelized island can be represented by the vertex coordinates of the six vertices of the minimum circumscribed polygon, that is, the ground projection position parameter α = (p α1 , p α2 , p α3 , p α4 , p α5 , p α6 ), image position parameter β=(p β1 , p β2 , p β3 , p β4 , p β5 , p β6 ). Among them, p α1 The coordinates p of the vertex α1 representing the minimum circumscribed polygon α1 =(x α1 ,y α1 ), the remaining parameters (p α2 …p α6 、p β1 …p β6 ) and p α1 The ground projection position parameter α=(x α1 ,y α1 , x α2 ,y α2 , x α3 ,y α3 , x α4 ,y α4 , x α5 ,y α5 , x α6 ,y α6 ), image position parameter β=(xβ1 ,y β1 , x β2 ,y β2 , x β3 ,y β3 , x β4 ,y β4 , x β5 ,y β5 , x β6 ,y β6 A set of location parameters represents a green belt or a channelized island. There are 24 parameters corresponding to a green belt or a channelized island.

[0067] The position parameters for guardrails, soundproof walls, water barriers, and fences are expressed in a similar manner to those for green belts and channelized islands. The ground projection position parameter α and image position parameter β of each guardrail, soundproof wall, water barrier, and fence can be represented by the coordinates of the four vertices of the minimum circumscribed polygon. A set of position parameters represents one guardrail, soundproof wall, water barrier, or fence, resulting in a total of 16 parameters corresponding to each guardrail, soundproof wall, water barrier, or fence.

[0068] It should be noted that the number of parameters for green belts, guardrails, soundproof walls, water barriers, fences, and channelized islands can be adjusted according to the number of vertices of the actual minimum circumscribed polygon, that is, the dimension of the position parameters can be designed according to actual needs, and there is no restriction here.

[0069] Combine Figure 3 To illustrate, Figure 3 A schematic diagram of stereoscopic target description in a scene information recognition method is provided for this application. Figure 3 The three-dimensional target in the figure is a green belt. The ground projection position parameter α and the image position parameter β of the green belt can also be represented by the vertex coordinates of the four vertices of the minimum circumscribed polygon. The red box in the figure represents the ground projection position of the green belt, and the yellow box represents the image position of the green belt. Therefore, the ground projection position parameter corresponding to the green belt is α = (p α1 , p α2 , p α3 , p α4 ), image position parameter β=(p β1 , p β2 , p β3 , p β4 ).

[0070] The remaining planar objects and stereoscopic objects that may exist in the scene image can be expanded using the same representation method as above, and there is no restriction here.

[0071] In another embodiment, the matching network may include a classification subnetwork, wherein the classification subnetwork can be used to obtain the category confidence of each object in the scene image. That is, the scene information can also include the category confidence of each object. After converting the semantic prediction results into scene information of the scene image, invalid objects whose category confidence is less than a confidence threshold can be found, and the category confidence and position parameters of the invalid objects in the scene information can be deleted.

[0072] In one specific embodiment, directly converting the semantic prediction results into the category confidence of each target in the scene image can be understood as a mapping process. For example, the semantic prediction results (category information, offset position, target information) of all pixels belonging to the same target are obtained using the target information in the semantic prediction results. The semantic prediction results are then used for mapping to obtain the category confidence of the target. The category confidence can be the confidence that the target belongs to each target category.

[0073] For example, the matching network includes a classification subnetwork and a regression subnetwork, and the scene information includes the position parameters of several targets in the scene image and the category confidence of each target. The classification subnetwork outputs K*M results, where K is the number of target categories and M is the preset maximum number of targets for each target category. Each output result represents the confidence of the Mth target belonging to the K category. The regression subnetwork can output normalized position parameters, whose dimension d O Designed as the sum of parameter dimensions for all target numbers where d k Represents the parameter dimension of the k-th category.

[0074] For example, the classification subnetwork outputs a 2*2 result, where the target categories are a and b. The confidence threshold for target category a is 0.8, corresponding to a parameter dimension of 3; the confidence threshold for target category b is 0.7, corresponding to a parameter dimension of 4. The classification subnetwork obtains the category confidences of target 1 belonging to category a (0.9) and b (0.1), and target 2 (0.2) and b (0.8). Based on the parameter format of the position parameters corresponding to different target categories, the regression subnetwork obtains the position parameters of target 1 belonging to category a and category b, and the position parameters of target 2 belonging to category a and category b. The total dimension of the output position parameters is 2*(3+4)=14. Invalid targets with category confidences less than the confidence threshold are identified and their category confidences and position parameters are deleted from the scene information. Based on the confidence thresholds for target class a and target class a, we know that target 1 belongs to target class a and target 2 belongs to target class a. The class confidence and corresponding position parameters for target 1 belonging to target class b are deleted, and the class confidence and corresponding position parameters for target 1 belonging to target class a are deleted. In other words, only the class confidence and corresponding position parameters for target 1 belonging to target class a and the class confidence and corresponding position parameters for target 2 belonging to target class b are retained.

[0075] In another embodiment, the scene information may be used as a scene reconstruction result of the target scene, or the category confidence and position parameters of each target in the scene information may be fused into the same image to obtain a scene reconstruction result of the target scene.

[0076] In a specific implementation, the scene information may be encapsulated and directly used as the scene reconstruction result.

[0077] In another specific embodiment, a multi-dimensional feature matrix can be constructed by analyzing the category confidence and position parameters of each target in the scene information. An adaptive weighted algorithm is used to map the category confidence and position parameters to a unified image coordinate system to form a composite feature map containing semantic information and spatial information. That is, the category confidence and position parameters are fused into the same image using the composite feature map to further obtain the scene reconstruction result of the target scene. Of course, the confidence and position parameters of each target in the scene information can also be fused into the same image in other ways, which can be designed according to actual needs and is not limited here.

[0078] In another embodiment, the scene information also includes the category confidence of each target. After converting the semantic prediction result into the scene information of the scene image, invalid targets whose category confidence is less than the confidence threshold can also be found, and the category confidence and the position parameters of the invalid targets in the scene information are deleted. The scene information is used as the scene reconstruction result of the target scene, or the category confidence and position parameters of each target in the scene information are fused into the same image to obtain the scene reconstruction result of the target scene. For specific instructions, please refer to the relevant description in step S13, which will not be repeated here.

[0079] See also Figure 4 , Figure 4 This is a flow chart of a specific embodiment of scene information recognition model training in the scene information recognition method provided by this application. Figure 4 As shown, this specific embodiment includes:

[0080] Step S41: Acquire several sample scene images.

[0081] The sample scene image is annotated with actual scene information of the sample scene image, and the actual scene information includes actual position parameters of each element target in the sample scene image.

[0082] In one embodiment, the actual scene information corresponding to the aforementioned plurality of sample scene images may be obtained from pre-annotated original annotation information.

[0083] In another embodiment, before acquiring several sample scene images, in response to the element targets in the sample scene images containing lines, a least squares method can be used to fit the sample fitting formula of the line, and the coordinates of the starting point and end point of the line in the sample scene image, as well as the sample fitting parameters in the sample fitting formula, can be obtained as the actual position parameters of the line. In response to the element targets in the sample scene images containing planar area targets, the coordinates of each sample vertex of the minimum circumscribed polygon of the planar area target can be detected using a convex hull detection method to serve as the actual position parameters of the planar area target. In response to the element targets in the sample scene images containing stereoscopic targets, the sample projection area of ​​the stereoscopic target on the ground plane and the sample contour area in the sample scene image are determined, and the coordinates of each sample vertex of the minimum circumscribed polygon of the sample projection area and the sample contour area are respectively detected using a convex hull detection method. The coordinates of each sample vertex corresponding to the sample contour area are used as the actual image position parameters in the actual position parameters of the stereoscopic target.

[0084] In one specific embodiment, the starting and ending coordinates of a line can be obtained from pre-annotated original annotation information, where the original annotation information can be a set of discrete pixels with an order. A sample fitting formula for the line can be obtained using the least squares method to obtain the sample fitting parameters corresponding to the line.

[0085] Specifically, there are N pixels in the original annotation information of the line. Taking the vertical line as an example, the sample fitting formula is x=θ0+θ 1y +θ2y 2 +θ3y 3 .

[0086] Construct the design matrix Y, where each row corresponds to a pixel and each column corresponds to a feature (i.e., the power of y):

[0087] Construct the observation vector x as:

[0088] Define the residual vector r as: r = YΘ-x.

[0089] The sum of the residual vector is: J(Θ)=r T r=(YΘ-x) T (YΘ-x).

[0090] Minimize the residual vector sum: Θ * =minJ(Θ).

[0091] Make its partial differential equal to 0, and solve to get the best estimated parameter: Θ * =(Y T Y) -1 Y T x.

[0092] The lane line parameter Θ=Θ can be obtained by the above formula * (θ0, θ1, θ2, θ3).

[0093] Similarly, for the horizontal lines, we solve the parameter Θ * =(X T X) -1 X T y, I will not go into details here.

[0094] In another specific embodiment, convex hull detection can be performed using Jarvis stepping method, Graham scanning, Andrew algorithm, QuickHull or Chan algorithm. For example, the pixel point at the lower left corner of the region is obtained from the original annotation information. The pixel point with the smallest y coordinate among all the pixel points contained in the region is selected as the starting point p0, wherein, when there are multiple pixel points with the same corresponding y coordinate, the pixel point with the smallest corresponding x coordinate is selected as the starting point p0. The remaining pixel points are sorted according to the polar angle relative to the starting point p0, wherein the polar angle refers to the angle between the vector from the starting point p0 to a remaining pixel point and the positive x-axis. When the polar angles of two pixel points are the same, the pixel point closer to the starting point p0 is placed in front. For example, the sorted pixel point set is {p1, p2, p3…p i}. Create a stack and push the starting point p0 and the first pixel point p1 after sorting into the stack. Starting from the second pixel point p2 after sorting, check each pixel point p in turn. i . Calculate the two pixels at the top of the stack and the current pixel p i When the above vector product is positive (indicating a counterclockwise direction), the pixel point p i Push it into the stack. If the vector product is negative (indicating a clockwise direction), pop the stack vertex and continue to check the new stack vertex and pixel point p i Repeat the above steps until the vector product is positive or there are only two pixels left in the created stack. The pixels in the stack are used as the vertices of the convex hull and output in order. That is, the vertices of the convex hull output in order are the sample vertices of the minimum circumscribed polygon.

[0095] Step S42: using a number of sample scene images to train the semantic network and matching network in the scene information recognition model.

[0096] It can be understood that based on the difference between the actual scene information of several sample scene images and the corresponding scene information of several sample scene images output by the scene information recognition model, the corresponding network parameters in the semantic network and the matching network in the scene recognition model can be adjusted to achieve training of the semantic network and the matching network in the scene recognition model.

[0097] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of the scene information recognition model training method provided by this application. The scene information recognition model can be used to implement the scene information recognition method in steps S11-S13. It should be noted that if there are substantially the same results, this embodiment does not use Figure 5 The process sequence shown is limited. Figure 5 As shown, this embodiment includes:

[0098] Step S51: Acquire several sample scene images.

[0099] The sample scene image is annotated with the actual semantic result and actual scene information of the sample scene image. The actual semantic result includes the actual category information of different pixels in the sample scene image, and the actual scene information includes the actual position parameters of each element target in the sample scene image. For a detailed description of the actual category information and actual position parameters, please refer to the relevant descriptions in steps S12 and S13 and are not repeated here.

[0100] In one embodiment, the actual semantic result may further include the actual offset positions of different pixels in the sample scene image. For detailed description of the actual offset positions, please refer to the relevant description of the offset positions in step S12, which will not be repeated here.

[0101] In another embodiment, the actual semantic result may further include actual target information of different pixels in the sample scene image. For a detailed description of the actual target information, please refer to the description of the target information in step S12, which will not be repeated here.

[0102] In another embodiment, the actual scene information may further include the actual category confidence of each target in the scene image. For a detailed description of the actual category confidence, please refer to the description of the category confidence in step S12, which will not be repeated here.

[0103] Step S52: Perform semantic prediction on the sample scene image using the semantic network of the scene information recognition model to obtain a sample semantic prediction result of the scene image.

[0104] Among them, the sample semantic prediction results include the predicted category information of different pixels in the scene image.

[0105] In one embodiment, the sample semantic prediction result further includes a predicted offset position. Detailed descriptions of the predicted offset position can be found in the description of the offset position in step S13 and will not be repeated here.

[0106] In another embodiment, the sample semantic prediction result further includes prediction target information. For detailed description of the prediction target information, please refer to the description of the target information in step S13, which will not be repeated here.

[0107] Step S53: using the matching network of the scene information recognition model to convert the sample semantic prediction result into sample scene information of the sample scene image.

[0108] The sample scene information includes sample position parameters of several targets in the scene image.

[0109] In one embodiment, the sample scene information further includes sample category confidence.

[0110] The specific method of obtaining the sample position parameters and the sample category confidence can refer to the relevant description in step S13, which will not be repeated here.

[0111] For the detailed description of step S53, please refer to the relevant description of step S13, which will not be repeated here.

[0112] Step S54: Based on the difference between the actual semantic result and the sample semantic prediction result, and the difference between the actual scene information and the sample scene information, the semantic network and the matching network of the scene information recognition model are adjusted.

[0113] The network parameters in the semantic network and matching network can be adjusted separately to optimize the performance of the scene information recognition model.

[0114] In a specific embodiment, the network parameters of the semantic network and the matching network can be adjusted based on the semantic network loss value corresponding to the semantic network and the matching network loss value corresponding to the matching network. For example, the loss value of the semantic network can be based on the loss function Loss s Get, among which, Loss s It is composed of the classification loss value corresponding to the target category, the regression loss value corresponding to the offset position, and the embedding feature loss value. The classification loss value can be obtained using the cross entropy loss function, and the regression loss value and the embedding feature loss value can be obtained using the L2 loss function. The loss value of the matching network can be based on the loss function Loss m Get, among which, Loss m It is composed of the classification loss value of the target category confidence and the regression loss value of the location parameter. The classification loss value can be obtained using the cross entropy loss function, and the regression loss value can be obtained using the L2 loss function.

[0115] The training of the scene information recognition model can be completed when the semantic network loss value corresponding to the semantic network and the matching network loss value corresponding to the matching network converge. The total loss value of the scene information recognition model can also be determined based on the semantic network loss value corresponding to the semantic network and the matching network loss value corresponding to the matching network, and the training of the scene information recognition model can be completed when the total loss value converges. In a specific embodiment, the back propagation algorithm is used to perform joint network training, wherein the total loss value of the network training can be calculated using the total loss function Loss. sum Get, the total loss function Loss sum As follows: Loss sum =Loss s +Loss m That is, by adjusting the network parameters of the semantic network and the matching network, the total loss value of the network training is converged.

[0116] See Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of the scene information recognition device of the present application. The scene information recognition device 600 includes a scene image acquisition module 610, a semantic prediction result acquisition module 620, and a scene information acquisition module 630. Among them, the scene image acquisition module 610 is used to obtain the scene image obtained by collecting the target scene. The semantic prediction result acquisition module 620 is used to perform semantic prediction on the scene image using the semantic network of the scene information recognition model to obtain the semantic prediction result of the scene image, wherein the semantic prediction result includes the category information of different pixels in the scene image, and the category information represents the target category to which the pixel belongs. The scene information acquisition module 630 is used to convert the semantic prediction result into scene information of the scene image using the matching network of the scene information recognition model, wherein the scene information includes the position parameters of several targets in the scene image, wherein the parameter format of the position parameters is related to the attributes of the target.

[0117] In some embodiments, several targets are at least one of planar targets and stereoscopic targets, the position parameters of the planar target are composed of the coordinates of at least two first position points of the planar target, the position parameters of the stereoscopic target include ground projection position parameters and image position parameters, the ground projection position parameters are composed of the coordinates of at least two second position points of the projection area of ​​the stereoscopic target on the ground plane, and the image position parameters are composed of the coordinates of at least two third position points of the contour area of ​​the stereoscopic target in the scene image.

[0118] In some embodiments, when the planar object is a planar linear object, the at least two first location points include the start point and the end point of the planar linear object; when the planar object is a planar area object, the at least two first location points include at least two vertices of the minimum circumscribed polygon of the planar area object. The at least two second location points include at least two vertices of the minimum circumscribed polygon of the projection area. The at least two third location points include at least two vertices of the minimum circumscribed polygon of the outline area.

[0119] In some embodiments, the target scene is a traffic scene, the plane linear target includes at least one of a line and an arrow, the line includes a lane line, the arrow includes a turn arrow, the plane area target includes at least one of the zebra crossing area, the grid area, the diversion area, and the waiting area, and the three-dimensional target includes at least one of a green belt, a guardrail, a soundproof wall, a water barrier, a fence, and a channelized island.

[0120] In some embodiments, the planar linear target includes a line, and the position parameters of the line further include fitting parameters in a fitting formula of the line.

[0121] In some embodiments, the planar linear target includes an arrow, and the position parameters of the arrow include the coordinates of the starting point, the middle point, and the end point of the arrow.

[0122] In some embodiments, the position parameters of the planar area type target include coordinates of vertices of a minimum circumscribed polygon of the planar area type target, and the shape of the minimum circumscribed polygon of the planar area type target is related to the attributes of the planar area type target.

[0123] In some embodiments, the ground projection position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the projection area of ​​the stereoscopic target, the image position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the contour area of ​​the stereoscopic target, and the shape of the minimum circumscribed polygon corresponding to the stereoscopic target is related to the attributes of the stereoscopic target.

[0124] In some embodiments, the semantic prediction result also includes at least one of the offset position of the different pixel points and the target information of the different pixel points, the offset position is used to indicate the position of the pixel point in the scene image, and the target information is used to determine the pixel points belonging to the same target.

[0125] In some embodiments, the category information is the confidence that the pixel belongs to each target category. The semantic prediction result is a semantic feature map with semantic information. The size of the semantic feature map is a scaled version of the scene image, and the offset position includes the offset of the vector from the pixel to the scaling starting point in each coordinate axis direction. The target information is an embedded feature.

[0126] In some embodiments, the scene information further includes a category confidence score for each of the objects. The scene information recognition device 600 includes a screening module configured to, after converting the semantic prediction results into scene information of the scene image, identify invalid objects whose category confidence scores are less than a confidence threshold, and delete the category confidence scores and location parameters of the invalid objects from the scene information.

[0127] In some embodiments, the scene information further includes a category confidence score of each target. The scene information recognition device 600 further includes a screening module configured to, after converting the semantic prediction result into scene information of the scene image, use the scene information as a scene reconstruction result of the target scene, or fuse the category confidence score of each target in the scene information and the position parameter into the same image to obtain a scene reconstruction result of the target scene.

[0128] In some embodiments, the training step of the scene information recognition model includes: obtaining a number of sample scene images, wherein the sample scene images are annotated with actual scene information of the sample scene images, wherein the actual scene information includes actual position parameters of each element target in the sample scene images; and using the sample scene images to train the semantic network and matching network in the scene information recognition model.

[0129] In some embodiments, the semantic network includes a feature extraction subnetwork, a feature fusion subnetwork, and a semantic prediction subnetwork. The semantic prediction result acquisition module 620 specifically executes the semantic prediction of the scene image using the semantic network of the scene information recognition model to obtain the semantic prediction result of the scene image, including: using the feature extraction subnetwork to extract features from the scene image to obtain a multi-scale scene feature map. Using the feature fusion subnetwork to fuse the multi-scale scene feature map to obtain a fused feature map. Using the semantic prediction subnetwork to perform semantic prediction based on the fused feature map to obtain the semantic prediction result of the scene image.

[0130] In some embodiments, before acquiring several sample scene images, the method further includes, in response to an element target in the sample scene image containing a line, using the least squares method to fit a sample fitting formula for the line, obtaining the coordinates of the starting point and end point of the line in the sample scene image, and the sample fitting parameters in the sample fitting formula as the actual position parameters of the line. In response to an element target in the sample scene image containing a planar regional target, using a convex hull detection method to detect the coordinates of each sample vertex of the minimum circumscribed polygon of the regional target as the actual position parameters of the planar regional target. In response to an element target in the sample scene image containing a 3D target, determining the sample projection area of ​​the 3D target on the ground plane and the sample contour area in the sample scene image, respectively detecting the coordinates of each sample vertex of the minimum circumscribed polygon of the sample projection area and the sample contour area using the convex hull detection method, and using the coordinates of each sample vertex corresponding to the sample contour area as the actual image position parameters in the actual position parameters of the 3D target.

[0131] See also Figure 7 , Figure 7: is a schematic diagram of a framework of an embodiment of a scene information recognition device of the present application. The scene information recognition device 70 includes a memory 71 and a processor 72 coupled to each other. The processor 72 is used to execute program instructions stored in the memory 71 to implement the steps of any of the above-mentioned scene information recognition method embodiments or to implement the steps of any of the above-mentioned scene information recognition model training method embodiments. In a specific implementation scenario, the scene information recognition device 70 may include, but is not limited to: a microcomputer, a server. In addition, the scene information recognition device 70 may also include mobile devices such as laptops and tablet computers, which are not limited here.

[0132] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above-mentioned scene information recognition method embodiments. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by an integrated circuit chip.

[0133] See also Figure 8 , Figure 8 Schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 80 stores program instructions 81 executable by a processor, which are used to implement the steps of any of the aforementioned scene information recognition method embodiments or the steps of any of the aforementioned scene information recognition model training method embodiments.

[0134] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0135] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0137] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0138] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A scene information recognition method, characterized in that: include: Acquire a scene image acquired from a target scene; Performing semantic prediction on the scene image using a semantic network of a scene information recognition model to obtain a semantic prediction result of the scene image, wherein the semantic prediction result includes category information of different pixels in the scene image, and the category information represents the target category to which the pixel belongs; The semantic prediction result is converted into scene information of the scene image using a matching network of the scene information recognition model, wherein the scene information includes position parameters of several targets in the scene image, wherein a parameter format of the position parameters is related to the attributes of the targets.

2. The method according to claim 1, characterized in that The several targets are at least one of planar targets and stereoscopic targets, the position parameters of the planar target are composed of the coordinates of at least two first position points of the planar target, the position parameters of the stereoscopic target include ground projection position parameters and image position parameters, the ground projection position parameters are composed of the coordinates of at least two second position points of the projection area of ​​the stereoscopic target on the ground plane, and the image position parameters are composed of the coordinates of at least two third position points of the contour area of ​​the stereoscopic target in the scene image.

3. The method according to claim 2, characterized in that When the planar target is a planar linear target, the at least two first position points include a start point and an end point of the planar linear target; when the planar target is a planar area target, the at least two first position points include at least two vertices of a minimum circumscribed polygon of the planar area target; The at least two second position points include at least two vertices of the minimum circumscribed polygon of the projection area; The at least two third position points include at least two vertices of a minimum circumscribed polygon of the contour area.

4. The method according to claim 3, characterized in that The target scene is a traffic scene, the plane linear target includes at least one of a line and an arrow, the line includes a lane line, the arrow includes a turn arrow, the plane area target includes at least one of a zebra crossing area, a grid area, a diversion area, and a waiting area, and the three-dimensional target includes at least one of a green belt, a guardrail, a soundproof wall, a water barrier, a fence, and a channelized island; And / or, the planar linear target includes a line, and the position parameter of the line also includes a fitting parameter in a fitting formula of the line; And / or, the planar linear target includes an arrow, and the position parameters of the arrow include the coordinates of the starting point, the middle point, and the end point of the arrow; And / or, the position parameters of the planar area type target include coordinates of each vertex of the minimum circumscribed polygon of the planar area type target, and the shape of the minimum circumscribed polygon of the planar area type target is related to the attributes of the planar area type target; And / or, the ground projection position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the projection area of ​​the stereoscopic target, the image position parameters of the stereoscopic target include the coordinates of each vertex of the minimum circumscribed polygon of the contour area of ​​the stereoscopic target, and the shape of the minimum circumscribed polygon corresponding to the stereoscopic target is related to the attributes of the stereoscopic target.

5. The method according to claim 1, wherein The semantic prediction result also includes at least one of the offset position of the different pixel points and the target information of the different pixel points, the offset position is used to indicate the position of the pixel point in the scene image, and the target information is used to determine the pixel points belonging to the same target.

6. The method according to claim 5, characterized in that The category information is the confidence level that the pixel belongs to each target category; The semantic prediction result is a semantic feature map with semantic information, the size of the semantic feature map is obtained by scaling the scene image, and the offset position includes the offset of the vector from the pixel point to the scaling starting point in the direction of each coordinate axis; The target information is an embedded feature.

7. The method according to claim 1, characterized in that The scene information also includes the category confidence of each target; After converting the semantic prediction result into the scene information of the scene image, the method further includes: Finding invalid targets whose category confidence is less than a confidence threshold, and deleting the category confidence and the position parameters of the invalid targets in the scene information; and / or, The scene information is used as a scene reconstruction result of the target scene, or the category confidence and the position parameters of each target in the scene information are fused into the same image to obtain a scene reconstruction result of the target scene.

8. The method according to claim 1, characterized in that The training steps of the scene information recognition model include: Acquire a plurality of sample scene images, wherein the sample scene images are annotated with actual scene information of the sample scene images, and the actual scene information includes actual position parameters of each element target in the sample scene images; Using the sample scene images to train the semantic network and the matching network in the scene information recognition model; And / or, the semantic network includes a feature extraction subnetwork, a feature fusion subnetwork, and a semantic prediction subnetwork; the semantic network using the scene information recognition model performs semantic prediction on the scene image to obtain a semantic prediction result of the scene image, including: Performing feature extraction on the scene image using the feature extraction subnetwork to obtain a multi-scale scene feature map; Using the feature fusion subnetwork to fuse the multi-scale scene feature map to obtain a fused feature map; A semantic prediction subnetwork is used to perform semantic prediction based on the fused feature map to obtain a semantic prediction result of the scene image.

9. The method according to claim 8, characterized in that Before acquiring a plurality of sample scene images, the method further includes: In response to the element target in the sample scene image including a line, a sample fitting formula of the line is obtained by using a least squares fitting method, and the coordinates of the starting point and the end point of the line in the sample scene image and the sample fitting parameters in the sample fitting formula are obtained as actual position parameters of the line; In response to the element targets in the sample scene image including a planar area type target, using a convex hull detection method to detect the coordinates of each sample vertex of the minimum circumscribed polygon of the planar area type target as the actual position parameters of the planar area type target; In response to the fact that the element targets in the sample scene image include a stereoscopic target, a sample projection area of ​​the stereoscopic target on the ground plane and a sample contour area in the sample scene image are determined, and the coordinates of each sample vertex of the minimum circumscribed polygon of the sample projection area and the sample contour area are detected respectively using a convex hull detection method, and the coordinates of each sample vertex corresponding to the sample contour area are used as actual image position parameters in the actual position parameters of the stereoscopic target.

10. A method for training a scene information recognition model, characterized in that: include: Acquire a plurality of sample scene images, wherein the sample scene images are annotated with actual semantic results and actual scene information of the sample scene images, the actual semantic results including actual category information of different pixels in the sample scene images, and the actual scene information including actual position parameters of each element target in the sample scene images; Performing semantic prediction on the sample scene image using a semantic network of a scene information recognition model to obtain a sample semantic prediction result of the scene image, wherein the sample semantic prediction result includes prediction category information of different pixels in the scene image; Converting the sample semantic prediction result into sample scene information of the sample scene image using a matching network of the scene information recognition model, wherein the sample scene information includes sample position parameters of a plurality of objects in the scene image; Based on the difference between the actual semantic result and the sample semantic prediction result, and the difference between the actual scene information and the sample scene information, the semantic network and the matching network of the scene information recognition model are adjusted.

11. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the scene information recognition method according to any one of claims 1 to 9 or the training method of the scene information recognition model according to claim 10.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the scene information recognition method according to any one of claims 1 to 9 or the training method of the scene information recognition model according to claim 10.

Images