COMPUTER-IMPLEMENTED METHOD FOR DETECTING A NEW OBJECT IN THE INTERIOR OF A TRAIN

Description

[0001] The invention relates to a computer-implemented method for detecting an object in the interior of a public transport vehicle that is not part of the interior furnishings. The invention further relates to a computer configured to execute the computer-implemented method. The invention also relates to a computer program.

[0002] It is known in the prior art to check the interior of a train for the presence of people using a camera. This is necessary, for example, before the train is parked on a siding. The prior art includes US 2020 / 103354 A1 (2020-04-02); BRUNO SAUVALLE ET AL: "Autoencoder-based background reconstruction and foreground segmentation with background noise estimation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, December 15, 2021; WO 2021 / 258664 A1 (2021-12-30).

[0003] The object of the invention is to provide an improved method for determining a higher accuracy in the detection of an object in the interior of a public transport vehicle that is not part of the interior furnishings. This object is achieved by the proposed method, computer, and computer program.

[0004] Further developments of the invention are specified in the dependent claims.

[0005] A computer-implemented method for detecting an object inside a public transport vehicle is proposed. The object is defined as not being part of the vehicle's interior furnishings. For example, the object could be a person, an animal, or an item such as a bag. This method could be used, for instance, to check whether a person, animal, or item is still inside the vehicle before it is parked in a designated area. A camera captures an image of the vehicle's interior, and this image is then reconstructed using a trained neural network.

[0006] The neural network is designed as a convolutional autoencoder. It was trained using images of the interior of a public transport vehicle, excluding objects not part of the interior furnishings. A self-learning, i.e., unsupervised, learning method was used for training. The trained neural network is able to distinguish between images of the interior of a public transport vehicle without objects and images of the interior of a public transport vehicle containing objects not part of the interior furnishings.

[0007] The reconstructed image is compared to the original image. This comparison is performed pixel by pixel. When comparing pixels in the original image with pixels in the reconstructed image, differences in brightness and / or color are determined. Specifically, all pixels in the original image are compared with their corresponding pixels in the reconstructed image. A pixel in the reconstructed image is identified as a different pixel if, for example, the difference in brightness and / or color exceeds a certain threshold. This threshold could be, for instance, 5% of the brightness or color value of the pixel in the original image.An object inside a public transport vehicle is detected based on the differing pixels, for example if the number of pixels exceeds a predefined limit and / or the differing pixels are arranged in a specific shape or structure.

[0008] Depending on the selected configuration, groups of pixels from the captured image can also be compared with corresponding groups of pixels from the reconstructed image. When comparing the group of pixels from the captured image with the group of pixels from the reconstructed image, a difference in brightness and / or color between the groups of pixels is determined. Specifically, all pixels of the captured image are compared in groups with the respective identical groups of pixels in the reconstructed image. A group of pixels in the reconstructed image is identified as deviating pixels if, for example, the average difference in brightness and / or the average difference in color of the group of pixels exceeds a predefined threshold. The threshold could, for example, be 5% of the average brightness and / or the average color value of the group of pixels in the captured image.

[0009] A signal is issued upon detection of an object inside the public transport vehicle that is not part of the furnishings.

[0010] This method makes it possible to quickly and reliably check for the presence of objects inside a public transport vehicle using an automated process. The object could be, for example, a person, an animal, or another item such as a bag. The described method has the advantage of being precise and functioning reliably regardless of varying lighting conditions. Furthermore, the proposed method is easy to implement because the neural network was trained using a self-learning algorithm with training images. Therefore, it is not necessary to label the pixels of the training images individually, significantly simplifying the effort required to train the neural network.Tests have shown that typical images from a public transport camera, specifically those that do not show any objects inside the vehicle, can be used to train the algorithm. Therefore, images of empty public transport interiors—that is, images showing only the interior furnishings—can be used to train the neural network.

[0011] The neural network can be implemented as a computer program and / or as hardware. For example, the neural network can be designed as a folded autoencoder. Neural networks in the form of folded autoencoders are particularly suitable for creating reconstructed images. In contrast to the original image, the reconstructed image exhibits lower accuracy with regard to brightness values ​​and / or the contours of the objects depicted. Thus, the objects in the reconstructed image are less sharp or less precisely and linearly represented. Furthermore, it has been shown that training the neural network preferably only with images of interiors without objects that are not part of the interior furnishings increases the probability that the method will be able to recognize images of interiors with objects that are not part of the interior furnishings with greater accuracy.

[0012] Depending on the chosen implementation, for example, in a black and white image with grayscale values, a brightness difference between the pixels being compared can be used to detect a deviating pixel. If the captured image and the reconstructed image are in color, then the brightness and / or the color difference can be used to detect a deviating pixel.

[0013] If, for example, an object—that is, a person or an item—is detected inside the public transport vehicle using this method, the system emits a signal. This signal can then be used by the computer to issue an instruction to the person inside the vehicle via a loudspeaker. Furthermore, the signal can instruct an operator to use the camera to examine the object inside more closely. This allows the operator to decide whether the public transport vehicle can be stopped or whether the object should be removed from the vehicle by an operator.

[0014] In another embodiment, the difference image containing the differing pixels is fed to a trained classifier. This trained classifier is designed to use the differing pixels in the difference image to check the interior of a public transport vehicle for the presence of an object and to detect its presence. Using the trained classifier improves the quality of the process and increases the probability of detecting an object in the interior of the public transport vehicle that is not part of the vehicle's interior furnishings. The classifier can be implemented, for example, as a support vector machine or a trained neural network.Using the classifier, it is possible to train not only the number of pixels but also the shape of the arrangement of the outlier pixels when evaluating the interior and taking this into account when deciding whether an object that is not part of the interior furnishings of the public transport vehicle is located inside. This increases the probability of correctly identifying an object that is not part of the interior furnishings of the public transport vehicle.

[0015] In another embodiment, the image captured by the camera is reduced in size before further processing, i.e., the number of pixels is decreased. At least a portion of the image that lies outside the potential area where a person could be inside the vehicle is removed. In this way, the number of pixels can be reduced without compromising the accuracy of the method for detecting a person inside the public transport vehicle. The reduced-size image is then used as a pre-processed image for the subsequent process. Preferably, the neural network is also trained using training images that are smaller than the image captured by the camera. This reduces processing time and saves storage space.

[0016] In a further embodiment, the captured image is preprocessed by removing at least one edge region of the image. In particular, the edge region can affect only one side of a rectangular image, or more specifically, two or all sides of the rectangular image. Depending on the chosen embodiment, edge regions of varying widths can be cropped from the different sides of the captured image before further processing. This allows for an image design optimized for the camera's image capture situation. Consequently, processing time and / or storage space can be saved.

[0017] In another embodiment, the captured image is pre-processed by removing areas of the image that lie above a predetermined height of the interior of the public transport vehicle. This predetermined height can be calculated as between 1.5 m and 2 m from the floor of the vehicle's interior. Tests have shown that, for person recognition using the described method, it is sufficient if the image depicts the interior from the floor up to a predetermined height of, for example, 1.5 m to 2 m. Areas of the image above this height are not required for person recognition. This also saves processing time and storage space.

[0018] In another embodiment, a trained second neural network is used as a classifier. This second neural network was trained using a supervised learning method to recognize an interior containing an object that is not part of the interior furnishings of the public transport vehicle, using difference images with objects that do not belong to the interior furnishings of the public transport vehicle and difference images without objects that do not belong to the interior furnishings of the public transport vehicle.

[0019] In a further embodiment, the first neural network was trained with images of interiors devoid of objects that are not part of the interior furnishings of the public transport vehicle in order to generate a reconstructed image from a captured image. Preferably, pre-processed images were also used for training, which are smaller in size than the images actually captured by the camera. For example, areas of the pre-processed images, particularly the edges, were removed. This results in the pre-processed image having fewer pixels. Thus, processing time and storage requirements are also saved during training.

[0020] In a further embodiment, an object that is not part of the interior furnishings of the public transport vehicle is detected based on the number of deviating pixels and / or the positions of these pixels. In addition to the number of deviating pixels, their positions can also indicate the presence of an object that is not part of the interior furnishings. This increases the probability that an object, in particular a person, an animal, or an item, will be reliably detected using the described method.

[0021] The properties, features and advantages of this invention described above, as well as the manner in which they are achieved, will become clearer and more easily understood in connection with the following description of the exemplary embodiments, which are explained in more detail in conjunction with the drawings, wherein FIG 1 in a schematic representation a car of a public transport vehicle, FIG 2 a first image from a camera in the car, FIG 3 a pre-processed first image of the FIG 2 , FIG 4 a second picture of another car, FIG 5 a pre-processed second image of the FIG 4 , FIG 6 a third picture of another car, FIG 7 a pre-processed third image of the FIG 6 , FIG 8 in a schematic representation a part of the computer that executes the procedure, FIG 9 a schematic representation of the procedure, FIG 10 one embodiment of the classifier as a neural network, FIG 11 in a left column of difference images with people and a right column of difference images without people, and FIG 12 A diagram showing the representation of deviating pixels for numbered test images shows.

[0022] FIG 1 Figure 1 shows a schematic representation of a car 1 of a public transport vehicle, which has an interior space 3. The public transport vehicle can z.B. It could be a train, a bus, a subway, or a tram. Inside car 1, interior 3 shows two seats, 2 and 52. A person is sitting in seat 2. 6. A camera 4 is provided on a ceiling 53 of the interior space 3, which, in the illustrated embodiment, takes a picture of the interior space 3 from above. The camera 4 is z.B. A digital camera, in particular a CCTV camera. The camera 4 is connected to a computer 5, which in the illustrated embodiment is arranged between a floor 33 of the interior 3 and the second seat 52. The connection between the camera 4 and the computer 5 can be wireless or via an electrical cable. The computer 5 has a processor and program and data memory.

[0023] FIG 2 Figure 7 is shown in a schematic diagram. It depicts a first image 7, captured by camera 4 of interior space 3. Due to the camera 4's optics, the first image 7 is square, with interior space 3 mapped onto a circular area. The first image 7 shows interior space 3 with the fixtures of the public transport vehicle, excluding any objects that are not part of the interior fixtures. The fixtures include chairs, partitions, handrails, etc. The fixtures are permanently installed in vehicle 1. An object is defined as any object that is not part of the interior fixtures of vehicle 1. Examples of objects include people, animals, bags, bicycles, etc. The first image 7 can be captured as a black and white image with grayscale or as a color image. For example, a color image can be converted into a grayscale image for simplified further processing.A black and white image with grayscale values ​​can be converted. The first image (7) is stored, for example, in the computer's data memory. The conversion of the color image to a grayscale image can be performed by the computer (5) using appropriate software.

[0024] FIG 3 Figure 8 shows a pre-processed first image 8, which was created by the computer 5 based on the first image 7. The pre-processed first image 8 has a smaller image area than the first image 7. The pre-processed first image 8 is rectangular in shape. In the pre-processed first image 8, outer edge areas of the first image 7 were cropped. The number of pixels in the image can be reduced, for example, by 10% or more, and in particular by 20% or more. In the illustrated embodiment, a first longitudinal area 31 and a second longitudinal area 32, which run along opposite longitudinal sides of the vehicle 1, were cropped. For example, image areas of the first image 7 were cropped that are located above a predetermined height of, for example, 1.5 m to 2 m above the floor 33 of the vehicle 1.

[0025] Furthermore, transverse areas 34 and 35 of the image, which run along opposite transverse sides of the vehicle 1, were cropped. These transverse areas 34 and 35 represent portions of the image that lie above a predetermined height above the ground 33, for example, 1.5 m to 2 m. Depending on the chosen embodiment, the pre-processed first image 8 can also be cropped in other ways to reduce the image size, i.e., the number of pixels, without losing any information that would impair the recognition of an object, particularly a person in the vehicle 1. Tests have shown that this reduction of the first image 7 to the pre-processed first image 8 saves computing time and memory requirements without compromising the result of the method.

[0026] FIG 4 Figure 9 shows a second image taken by a camera from the interior of another public transport vehicle. Due to the camera's optics, the second image 9 has a rectangular shape. Furthermore, the interior of the other vehicle differs from the interior of vehicle 1. FIG 2 They are designed differently. The second image, number 9, can also be reduced in size. FIG 5 Figure 1 shows a pre-processed second image 10, which was created by the computer 5 based on the second image 9. Longitudinal areas 31, 32 and transverse areas 34, 35 of the second image 9 were cropped. In the illustrated example of the pre-processed second image 10, areas of the second image 9 that lie above a predetermined height relative to the floor 33 of the further carriage were cropped. Depending on the chosen embodiment, the pre-processed first image 8 can also be cropped in other ways to reduce the size of the second image, i.e., the number of pixels, without losing any information that would impair the recognition of an object, in particular a person, in the further carriage.

[0027] FIG 6 Figure 11 shows a third image, taken by a camera from the interior of another car. This third image shows a different interior than the first or second images 7 and 9. The third image 11 can also be reduced in size, analogous to the first and second images 7 and 9, by cropping out edge areas 31, 32, 34, and 35 of the third image 11, as shown in Figure 11. FIG 7 is represented in the form of the pre-processed third image 12.

[0028] The preprocessing of the images FIG 2 , 4 , 6 to the images of FIG 3 , 5The operation 7 can be performed either directly in the camera 4 by a corresponding processor with appropriate software, or by the computer 5, depending on the embodiment of the camera 4. Depending on the chosen embodiment, the longitudinal areas 31, 32 and / or transverse areas 34, 35 to be cut off can also be individually adjusted by an operator. Thus, the longitudinal areas 31, 32 and / or the transverse areas 34, 35 can also have surface shapes other than rectangular ones.

[0029] FIG 8 Figure 5 shows a schematic diagram of the structure of a part of computer 5, which can be used to execute the computer-implemented method for detecting an object inside a public transport vehicle. Computer 5 includes, for example, a first neural network 14, which in this example is configured as an autoencoder. The neural network 14 is implemented as a software program and / or as a hardware circuit.

[0030] The autoencoder is a neural network that learns an identity function unsupervised to reconstruct the original input, compressing the data to find a more efficient and compact representation. The autoencoder consists of an encoder network and a decoder network. The encoder network translates the original high-dimensional input into a latent low-dimensional code. The decoder network extracts the data from the code. The decreasing number of neurons in the decoder network forces the autoencoder to reduce the dimensionality. This can only be achieved by the neural network learning to disregard unimportant information in the input image features.

[0031] The autoencoder contains an encoding function g parameterized by a first parameter ϕ and a decoding function f parameterized by a second parameter θ. The low-dimensional code learned for the input x in the compression layer 46 is z=gϕ(x) and the reconstructed input at the output of the output layer is x'=fθ(gϕ(x)).

[0032] The parameters (θ,ϕ) are learned together to generate an image 15 reconstructed from the supplied image 13, which is identical to the supplied image x≈fθ(gϕ(x)), or in other words, the autoencoder is trained to learn an identity function.

[0033] There are various metrics to quantify the difference between two images, e.g., cross-entropy when the activation function is sigmoid, or simply MSE loss (LAE). LAE θ ϕ = ln ∑ i = 1 n x i − fθ gϕ x i 2

[0034] Autoencoders are trained in the same way as a normal neural feedforward network. As with the normal neural network, the layers of the autoencoder are connected to each other by weights.

[0035] For an input vector x, a product is formed between the input vector x and the weight matrix W, which connects the input layers to the first hidden layer. After this step, a nonlinear function such as Tanh ReLU or Sigmoid is applied to the result of the product.

[0036] This process is repeated for each layer until the compression layer is reached. Here, the hidden neurons are typically represented by the vector z, which is the latent representation of the input vector x. This was the coding step, which can be summarized by the following equations: a 1 → = σ W 1 ∗ x → z → = σ W 2 ∗ a 1 →

[0037] Here, the vector represents xthe input vector with the input values, the vector a 1 the values ​​of the neurons in the first hidden layer of the neural network and the vector z The values ​​of the neurons in the compression layer are represented. Sigma ( σ ) is an arbitrary nonlinear activation function. W1, W2 are the respective weight matrices.

[0038] During the decoding step, the latent vector z This is used as a new input, and the same process as before is repeated: calculating products and applying activation functions. The final output of the network is the vector. x → ^ , which is the reconstructed input of the vector x This represents the decoding. The decoding is described by the following equations: a 2 → = σ W 3 ∗ z → x → ^ = σ W 4 ∗ a 2 →

[0039] As mentioned previously, the autoencoder is trained to calculate the difference between the input vector xand its reconstructed counterpart x → ^ is reduced, where W3 and W4 represent corresponding matrices, and where a 2 represents the vector after the compression layer. This goal can be formulated as minimizing the loss function of the mean squared error. The minimization of the loss function can be achieved here by regular gradient descent. L θ = 1 N ∑ i = 0 N x ι → − x ι → ^ 2

[0040] Minimizing the MSE loss function L ( θ ) reduces the difference between the input vector x and the initial vector x → ^ This automatically leads to better encoding of the input vector. x .

[0041] A further image 13 is fed to the neural network 14 on an input layer 41. In the illustrated embodiment, the first neural network 14 is configured as an autoencoder. The further image 13 is supplied by the camera 4. For example, the further image 13 can be an image captured by the camera according to the FIG 2 , 4 , 6 Furthermore, depending on the chosen embodiment, the additional image 13 can be a pre-processed image based on the image captured by the camera, according to the FIG 3 , 5 , 7 be trained.

[0042] The input layer 41 of the neural network has, for example, as many input neurons 42 as the subsequent image 13 has image pixels. Each image pixel is fed to an input neuron 42. Each input neuron 42 is connected to each neuron 43 of a first hidden layer 44. Furthermore, each neuron 43 of the first hidden layer 44 is connected to each neuron 45 of the compression layer 46. The first hidden layer 44 has fewer neurons 43 than the input layer 41. The compression layer 46 has fewer neurons 45 than the first hidden layer 44.

[0043] Each neuron 45 of compression layer 46 is connected to each neuron 47 of a second hidden layer 48. The second hidden layer 48 has more neurons than compression layer 46. For example, the second hidden layer 48 can have the same number of neurons as the first hidden layer 44. Each neuron 47 of the second hidden layer 48 is connected to a neuron 49 of output layer 50. The number of neurons 49 of output layer 50 can be equal to the number of neurons 42 of input layer 41.

[0044] Depending on the chosen implementation, the neural network 14 can also exhibit a different form of autoencoder. Further examples of autoencoders include, for example, convolutional autoencoders, as described in Chapter 3.3 Encoder Decoder Based Models in "Image Segmentation Using Deep Learning: A Survey" by Shervin Minae et al., ar Xiv:2001.05566v5.

[0045] The neural network 14, d.h. In the illustrated embodiment of the autodecoder, it has been trained in an unsupervised learning process such that the output layer 50 creates a reconstructed further image 15 of the further image 13. For the unsupervised learning of the neural network 14, images from the cameras can be used according to the FIG 2 , 4 , 6 or pre-processed images from the cameras according to the FIG 3 , 5 or 7 images of interiors without objects are preferably used for training purposes. d.h. The images only show the interior of the public transport vehicle, without any object being inside. d.h. a bag, an animal, or a person is located. For example, a neural network 14 is trained for each camera, using only images of the interior without objects that are not part of the furnishings. These images were taken under various lighting conditions. Thus, each neural network 14 is trained precisely on the optical representation of the respective interior. Depending on the chosen implementation, a neural network 14 can also be trained with images of different interiors, preferably each containing no objects that are not part of the interior furnishings of the public transport vehicle. This allows for the creation of reconstructed images for different interiors, saving the number of neural networks required.

[0046] Neural network 14 is trained to reconstruct an image of an interior space without any objects that are not part of the interior's furnishings. Therefore, the quality of the image reconstructed by the neural network of an interior space containing an object that is not part of the interior's furnishings is lower, as this reconstruction was not trained. Consequently, the number of differing pixels when creating a difference image between a camera image and the image reconstructed by the neural network based on the camera image is significantly higher than when creating a difference image between an image of an interior space without an object that is not part of the furnishings and an image reconstructed by the trained neural network based on the camera image. Therefore, the trained neural network 14 is very well suited to detecting an interior space containing an object that is not part of the interior's furnishings.

[0047] The reconstructed additional image 15 and the additional image 13 are then fed to a processing unit 16. The processing unit 16 determines a difference image between the additional image 13 and the reconstructed additional image 15. In doing so, the reconstructed additional image 15 is compared with the captured additional image 13, with the comparison being performed pixel by pixel. During this comparison, a brightness difference and / or a color difference between the pixels of the captured additional image 13 and the reconstructed additional image 15 is determined. Thus, difference images are obtained, as for example in FIG 11 This is shown. The difference images 20, 22, 24 in the right column of the FIG 11 Each image shows a difference view for the interior space 3 of a car 1 of a public transport vehicle, where no object, i.e., no item, animal, or person, is located in the interior space 3 of the car 1 other than the furnishings. In contrast, the left column of the FIG 11 A first, a third, and a fifth difference image 19, 21, 23, each determined for an image from a camera of the interior 3 of a car 1 containing an object that is not part of the interior furnishings of the public transport vehicle, and the image reconstructed on the basis of the camera image using the trained neural network 14. The object can z.B. The images depict one or two people. The difference images clearly show that the creation of the difference images results in distinct differences between images with and without objects. Each difference image displays only those pixels from the reconstructed additional image 15 and the recorded additional image 13 whose brightness and / or color differences exceed a predefined threshold.

[0048] The first, third and fifth difference images 19, 21, 23 of the left column of the FIG 11 The images show the same interior of the public transport vehicle under different lighting conditions. Similarly, the second, fourth, and sixth difference images 20, 22, 24 show the same interior of the public transport vehicle under different lighting conditions.

[0049] The difference image 19 created by processing unit 16 is then fed to a second processing unit 18. The second processing unit 18 can, for example, be a trained classifier.

[0050] The classifier is trained to recognize, based on a difference image, whether or not an object is located inside a public transport vehicle. For example, the classifier can be implemented as a second trained neural network 25, as in FIG 10 The second neural network 25 is illustrated. In the illustrated embodiment, the second neural network 25 has a first input layer 26 with neurons to which the individual image pixels of the difference image are fed. Each neuron of the input layer 26 is connected to each neuron of a hidden layer 27. The input layer 26 has, for example, as many neurons as the difference image 19 has image pixels. The hidden layer 27 can have more or fewer neurons than the input layer 26. Each neuron of the hidden layer 27 is connected to the single output neuron 28.The output neuron 28 outputs an output value 29, which indicates whether and with what probability the difference image is a difference image for the interior of a car with an object that is not part of the interior furnishings, or whether and with what probability the difference image is a difference image for the interior of a car without an object that is not part of the interior furnishings.

[0051] The second neural network 25, for example, was trained using a supervised learning method with appropriately labeled difference images to distinguish a difference image for an interior space with an object that is not part of the interior space's furnishings from a difference image for an interior space without an object that is not part of the interior space's furnishings.

[0052] The initial value 29 is converted into a corresponding signal 17 by computer 5 when a difference image of an interior space with an object that is not part of the interior furnishings is detected. Signal 17 can be output electrically, optically, or acoustically. For example, signal 17 can be issued as an announcement over a loudspeaker in the interior, requesting the person to leave the vehicle. Alternatively, signal 17 can be issued as an instruction to an operator, directing them to inspect the vehicle and remove the object.

[0053] Depending on the chosen embodiment, the second processing unit 18 can also count the pixels shown in the difference image 19. For example, it shows FIG 12 A diagram where the x-axis represents the number of the difference image and the y-axis represents the number of differing pixels. Additionally, a threshold th is marked on the y-axis at approximately 10 pixels. If the number of differing pixels exceeds the threshold th, the difference image is interpreted as an image of a car interior containing an object. Accordingly, the second processing unit 18 outputs a signal 17. If the number of differing pixels is below the threshold th, the difference image is interpreted as an image of an interior without an object, and no signal 17 is output. The threshold th of 10 pixels is only an example and can have other values.

[0054] FIG 9 Figure 1 shows a schematic diagram of the program sequence for carrying out the described procedure. At program point 100, camera 4 takes a picture of the interior 3 of car 1 of the public transport vehicle. Subsequently, in a following optional program step 105, the captured image 13 is preprocessed, whereby the size of the captured image 13 is reduced. Areas of the captured image 13 are removed that are irrelevant for checking whether an object, in particular a person, an animal, or an object that is not part of the interior furnishings of the vehicle, is present. At least one outer edge area of ​​the captured image 13 can be cropped, as shown in Figure 105. FIG 2 bis 7 This was explained. For example, peripheral areas can be removed from all four sides of the image. In particular, peripheral areas of an image can be removed that represent an area of ​​the interior that is above a specified height relative to the floor of the vehicle, especially above a specified height between 1.5 m and 2 m. This allows even shorter people to be recognized, even if parts of their upper body or head are no longer captured by the image.

[0055] In a subsequent program step 110, the recorded image 13 or the pre-processed image 13 is fed to a trained neural network 14. The trained neural network 14 is trained to generate a reconstructed image 15 from the fed image 13. The first neural network 14, for example, has the form of a folded autoencoder, as shown by FIG 8 This was explained. At a subsequent program point 120, the trained first neural network 14 outputs the reconstructed image 15.

[0056] Subsequently, in a following program 130, the captured image 13 and the reconstructed image 15 are compared, creating a difference image. During this comparison, the individual pixels of the captured image 13 are compared with the individual pixels of the reconstructed image with respect to brightness and / or color. If a pixel exceeds a predefined brightness threshold and / or color threshold, that pixel is recognized as a deviating pixel and displayed in the difference image. Non-deviating pixels are not shown in the difference image. Depending on the chosen implementation, the deviating pixel in the difference image can be displayed either as a black pixel on a white background or, depending on the difference value between the pixel of the captured image 13 and the pixel of the reconstructed image 15, with a gray value.

[0057] In the following program step 140, the difference image is analyzed and checked according to predefined procedures to determine whether the difference image represents a difference image of an interior space with an object that is not part of the interior furnishings of the public transport vehicle. Various methods can be used for this. For example, a second trained neural network or a trained classifier can be used. The second trained neural network or the classifier has been trained with difference images to distinguish a difference image of an interior space with an object that is not part of the interior furnishings of the public transport vehicle from a difference image of an interior space without such an object. For example, the second neural network can be trained according to the FIG 10 be trained. In addition, according to another method, the presence of a difference image for an interior space with an object can be recognized by a second processing unit 18 if the number of differing pixels in the difference image exceeds a predefined threshold th, as based on FIG 12 was explained.

[0058] At a subsequent program point 150, the computer 5 outputs a signal 17 if a difference image is detected for an interior space containing an object that is not part of the interior furnishings of the public transport vehicle. Depending on the selected embodiment, at program point 160, after outputting a signal 17, the computer can then issue a visual and / or audible signal in the interior 3 of the vehicle 1, for example, requesting a person to leave the vehicle. The signal can also be forwarded to an operator with the instruction to check the vehicle for an object or for the presence of an animal or person.

[0059] The program then branches back to point 100 and takes another picture, for example of another car. In this way, several cars of a public transport vehicle can be checked for the presence of an object by a computer 5, which is, for example, centrally located.

[0060] Although the invention has been illustrated and described in detail by the preferred embodiment, the invention is not limited by the disclosed examples and other variations can be derived by the person skilled in the art without leaving the scope of protection of the invention.

[0061] As used herein, a computer, classifier, processing unit 16, or second processing unit 18 corresponds, for example, to a processor or any electronic device configured via hardware circuitry, software, and / or firmware to process data. For example, the processors described herein may correspond to one or more (or a combination of) microprocessors, CPUs, or other integrated circuits (ICs), or other types of circuitry capable of processing data in a data processing system.It should be understood that a processor described or claimed to be configured to perform a particular described / claimed process or function may correspond to a CPU that executes computer / processor executable instructions stored in memory in the form of software and / or firmware to perform such a described / claimed process or function; and / or may correspond to an IC that is hardwired to a processing circuit (e.g., an FPGA or ASIC IC) to perform such a described / claimed process or function.

[0062] It should also be understood that a computer or classifier described or claimed to be configured to perform a particular described / claimed process or function may correspond to a combination of the computer or classifier with software instructions loaded / installed into the described memory (volatile and / or non-volatile) that are currently executing and / or available for execution by the computer or classifier to cause the computer or classifier to perform the described / claimed process or function.Thus, a processor that is switched off or running other software, but has installed the described software instructions on a storage device in operational association with it (such as a hard disk or SSD) in a manner configured to be executed by the computer or classifier (when started by a user, hardware and / or other software), may also correspond to the described / claimed computer or classifier configured to perform the special processes and functions described / claimed herein.

[0063] The memory of the computer or classifier can correspond to internal or external volatile memory (e.g., main memory, CPU cache, and / or RAM) contained within the computer and / or operationally connected to the computer or classifier. Such memory can also correspond to non-volatile memory (e.g., flash memory, SSD, hard disk, or other storage device or non-transient computer-readable medium) operationally connected to the computer.

[0064] The described computer or classifier may include at least one input device and at least one display or output device in operational connection with the computer. The input device may include, for example, a mouse, a keyboard, a touchscreen, a gesture input device, or any other type of input device capable of providing user input to the computer or classifier. The display device may include, for example, an LCD or AMOLED screen, a monitor, a head-mounted display, or any other type of display or output device capable of displaying output from the computer or classifier.For example, the computer or classifier, the memory, the software instructions, the input device and the display device may be included as part of a data processing system corresponding to a PC, a workstation, a server, a notebook, a tablet, a mobile phone, a head-mounted display or any other type of computer system or any combination thereof.

[0065] The computer or classifier may also include one or more data stores. The computer may be configured to manage, retrieve, create, use, modify, and store data and / or other information described herein from / in the data store. Examples of a data store may include a file and / or record stored in a database (e.g., Oracle, Microsoft SQL Server), a file system, a hard disk, an SSD, a flash drive, a memory card, and / or any other type of device or system that stores non-volatile data.

[0066] It should be noted that although the disclosure contains a description relating to a fully functional computer or classifier and / or a series of actions, those skilled in the art will understand that at least parts of the mechanism of the present disclosure and / or the described actions can be distributed in the form of executable computer / processor instructions (e.g., the described software instructions and / or the corresponding firmware instructions) contained in any form on a non-transitory machine-usable, computer-usable, or computer-readable medium, and that the present disclosure applies equally regardless of the specific type of instruction or data carrier medium or storage medium used to actually carry out the distribution.Examples of non-transitory machine-readable or computer-readable media include ROMs, EPROMs, magnetic tapes, hard disk drives, SSDs, flash memory, CDs, DVDs, and Blu-ray discs. Instructions executable by the computer / processor can comprise a routine, a subroutine, programs, applications, modules, libraries, and / or the like. Furthermore, it should be noted that the executable computer / processor instructions can be source code, bytecode, runtime code, machine code, assembly language, Java, JavaScript, Python, C, C#, C++, or any other form of code that can be programmed / configured to cause at least one processor to perform the actions and features described herein. Additionally, the results of the described / required processes or functions can be stored on a computer-readable medium, displayed on a display device, and / or the like.Neural networks and machine learning models can be implemented and used in a machine learning framework, such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0067] It should also be acknowledged that devices referred to as input devices or output devices can both provide input and receive output from the computer. For example, the computer may be integrated into a housing (such as a tablet or a head-mounted display) that contains a touchscreen or is capable of capturing motion gesture input, thus serving as both an input and display device. Furthermore, it should be noted that some input devices may encompass a variety of different input types (e.g., touchscreen, touchpad, and keyboard).

[0068] In addition to a communication controller, other devices may be connected to the computer, such as any local area network, wide area network, long-distance network, private and / or public data processing system, or a combination of networks known to a person skilled in the art, including the Internet.

[0069] Furthermore, it should be noted that data processing systems can contain virtual machines in a virtual machine architecture or in the cloud. For example, the computer and its associated components can correspond to the combination of one or more virtual machine processors of a virtual machine running on one or more physical processors of a physical data processing system.

[0070] It should also be noted that the computer described here can correspond to a remote processor located in a data processing system, such as a server, that is remote from the display and input devices described herein. In such an example, the described display and input devices can be contained within a client data processing system (which may have its own processor) that communicates with the server (containing the remote processor) over a wired or wireless network (which may include the internet). In some embodiments, such a client data processing system might, for example, run a remote desktop application or correspond to a portal device that executes a remote desktop protocol with the server to send input from an input device to the server and receive visual information from the server for display via a display device.Examples of such remote desktop protocols include Teradici's PCoIP, Microsoft's RDP, and the RFB protocol. In another example, such a client computing system might run a web browser or a thin client application. User input can be transmitted from the web browser or thin client application to the server for evaluation, rendering, and sending an image (or series of images) back to the client computing system for display by the web browser or thin client application. In some examples, the remote processor described here might also correspond to a combination of a virtual processor and a virtual machine running within a physical processor of the server.

[0071] As used herein, the terms "component" and "system" are intended to encompass hardware, software, or a combination of hardware and software. For example, a system or component could be a process, a process running on a processor, or a processor itself.

[0072] The statements regarding the computer and the classifier also apply to processing unit 15 and the second processing unit 16.

[0073] Furthermore, a component or system can be located on a single device or distributed across multiple devices.

[0074] None of the descriptions in the present application should be understood as indicating that any particular element, step, action, or function is an essential element that must be included in the scope of the claims: the scope of the patented subject matter is defined solely by the admitted claims. Furthermore, none of these claims should be understood as relying on a means-plus-function claim structure unless the exact words "means for" are followed by a participle.

Claims

1. Computer-implemented method for detecting an object in an interior of a public means of transport which is not part of the facility of the interior of the public means of transport, - wherein an image of the interior of the public means of transport is captured by a camera, and - wherein a signal is output when an object is detected in the interior of the public means of transport, characterised in that - the captured image is converted into a reconstructed image with the aid of a trained neural network, - wherein the trained neural network is embodied as an autoencoder, - wherein the neural network was trained using a self-learning method with images of the interior without objects, which are not part of the facility of the interior of the public means of transport, in order to generate a reconstructed image from a captured image, whereby the quality of the image, reconstructed by the neural network, of an interior with an object, which does not form part of the facility of the interior, is worse, - wherein the reconstructed image is compared with the captured image, - wherein a pixel-by-pixel comparison is performed when the captured image is compared with the reconstructed image, - wherein when the pixels to be compared of the captured image and of the reconstructed image are compared, a difference in brightness and / or a difference in colour between the pixels is determined, - wherein a pixel of the reconstructed image is detected as an anomalous pixel if the difference in brightness and / or the difference in colour exceeds a threshold value, - wherein an object in the interior of the public means of transport, which does not form part of the facility of the interior of the public means of transport, is detected on the basis of the anomalous pixels, - wherein a differential image, which has the anomalous pixels, is fed to a trained classifier, - wherein the classifier was trained in order to be able to differentiate an interior of a public means of transport with an object from an interior of a public means of transport without an object on the basis of the anomalous pixels of a differential image, - wherein on the basis of the differential image the classifier evaluates whether or not an object is located in the interior of the public means of transport, - wherein the classifier is embodied as a trained second neural network, - wherein the second neural network was trained with differential images with objects and with differential images without objects in order to detect an interior with an object.

2. Method according to claim 1, - wherein in order to train the classifier so as to differentiate between an image of an interior of a public means of transport without an object, which is not part of the facility of the public means of transport, and an image of an interior of a public means of transport with an object, which is not part of the facility of the public means of transport, supervised learning with differential images is used, - wherein a differential image is formed between a captured image and a reconstructed image, - wherein the differential image is determined by a pixel-by-pixel comparison of the captured image with the reconstructed image, - wherein a difference in brightness and / or a difference in colour between the pixels is determined when the pixels to be compared of the captured image and the reconstructed image are compared, - wherein a pixel of the reconstructed image is detected as an anomalous pixel and captured in the differential image if the difference in brightness and / or the difference in colour exceeds a predetermined threshold value.

3. Method according to one of the preceding claims, - wherein the size of the image captured by the camera is reduced in the number of pixels before further processing, - wherein at least one region of the image, which lies outside of an area possibly occupied by a person in the interior, is removed, and a pre-processed image is obtained, - wherein the method is carried out with the pre-processed image.

4. Method according to one of the preceding claims, - wherein an edge region of the captured image is removed during the pre-processing and a pre-processed image with fewer pixels is obtained as a result.

5. Method according to one of the preceding claims, - wherein before the processing of the captured image, regions of the captured image are removed, which display regions of the interior, which are above a predetermined height of the interior, - wherein the predetermined height is between 1.5 m and 2 m in height as measured from a floor of the interior.

6. Method according to one of the preceding claims, - wherein the object represents a person.

7. Method according to one of the preceding claims, - wherein pre-processed images were used for training, in which regions of the image were removed and as a result the pre-processed image has fewer pixels than the captured image.

8. Method according to one of the preceding claims, - wherein an object which is not part of the facility of the interior of the public means of transport is detected in the interior of the public means of transport on the basis of a number of anomalous pixels and / or on the basis of the positions of the anomalous pixels, - wherein a signal is output when the object which is not part of the facility of the interior of the public means of transport is detected.

9. Computer which is embodied to carry out one of the preceding methods.

10. Computer program which has commands which, upon execution on a computer, carry out a method according to one of claims 1 to 8.

11. Machine-readable storage medium, on which the computer program according to claim 10 is stored.